-- Logs begin at Tue 2022-08-30 23:31:58 UTC, end at Wed 2022-08-31 00:53:17 UTC. -- Aug 31 00:13:56.825566 dv-cinder-852211-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 00:13:56.834396 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 00:13:56 2022] *** Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: nodename: dv-cinder-852211-6-iscsi Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: machine: x86_64 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: clock source: unix Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: pcre jit disabled Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: detected number of CPU cores: 4 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: current working directory: / Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: detected binary path: /usr/bin/uwsgi-core Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: your processes number limit is 47680 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: your memory page size is 4096 bytes Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: detected max file descriptor number: 2048 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: lock engine: pthread robust mutexes Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: thunder lock: enabled Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 00:13:56.839172 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 00:13:56.878530 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: Python main interpreter initialized at 0x5582bd9629c0 Aug 31 00:13:56.878530 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: python threads support enabled Aug 31 00:13:56.878530 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: your server socket listen backlog is limited to 100 connections Aug 31 00:13:56.878530 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: your mercy for graceful operations on workers is 90 seconds Aug 31 00:13:56.878946 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 00:13:56.878946 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: *** Operational MODE: preforking *** Aug 31 00:13:56.879263 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: *** uWSGI is running in multiple interpreter mode *** Aug 31 00:13:56.879263 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: spawned uWSGI master process (pid: 88204) Aug 31 00:13:56.879503 dv-cinder-852211-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 00:13:56.881285 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: spawned uWSGI worker 1 (pid: 88218, cores: 1) Aug 31 00:13:56.881285 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: spawned uWSGI worker 2 (pid: 88219, cores: 1) Aug 31 00:13:56.881285 dv-cinder-852211-6-iscsi devstack@keystone.service[88204]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 00:14:01.335718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:14:01.335718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: from cryptography.utils import int_from_bytes Aug 31 00:14:01.336561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:14:01.336561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: from cryptography.utils import int_from_bytes Aug 31 00:14:01.336674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:14:01.336674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: from cryptography.utils import int_from_bytes Aug 31 00:14:01.336674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:14:01.336674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: from cryptography.utils import int_from_bytes Aug 31 00:14:01.388723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 00:14:01.388936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 00:14:01.389097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] command line args: [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 00:14:01.389370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 00:14:01.389550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 00:14:01.389650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ================================================================================ {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 00:14:01.390433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] admin_token = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.390569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 00:14:01.390709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] config_dir = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.390802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] command line args: [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 00:14:01.391021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.391113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 00:14:01.391313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ================================================================================ {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 00:14:01.391403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] config_source = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.391651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] debug = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.391754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] admin_token = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] config_dir = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] config_source = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] debug = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.392926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_config_append = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.393975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_config_append = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_dir = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_file = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_dir = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_file = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.394887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.395966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.396931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.397952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.398067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.398177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.398343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.398463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.398613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] publish_errors = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] publish_errors = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.399980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] standard_threads = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] standard_threads = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.400944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] use_journal = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] use_journal = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.401997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] use_json = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] use_json = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] use_stderr = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] use_stderr = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] use_syslog = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] use_syslog = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:14:01.402934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.403861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.404995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.405219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.405219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.405439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.405613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.405819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.406923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.407043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.407272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.407459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.407642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.407830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.407975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.408934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.409802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.410918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.411905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.412968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.413845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.414987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.415149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.415149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.415443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.415605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.415766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.415899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.416064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.416233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.416442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.416772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.416952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.417990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.418948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.419083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.419152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.419322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.419404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.419561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.419630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.420899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.421884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422581 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.422928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.423946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.424148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.424366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.424549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.424620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.424826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.424893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.425968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.426939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.external = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.427995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.password = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] auth.token = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.428986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.429937 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.430069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.430154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.430300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.430384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.430826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.431004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.431004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.431155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.431227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.431297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.432970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.external = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.password = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] auth.token = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.433971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.434870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.435985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.436189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.436355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.437950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.438907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.439997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.440127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.440540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.440655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.440801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.440929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.441972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.442898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.443954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.444932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.445909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.446952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.447939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.448877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.449991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.450945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.451916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.452034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.452205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.452854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.452968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.453873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.454929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.455977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.456898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.457939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458796 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.458920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.459992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.460091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.460234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.460424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.460560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.460739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.460889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] role.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] role.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.461930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.462888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.463975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.464096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.464264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.464746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.464928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.465055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.465196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.465380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.465380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.465718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.465891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.466962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.467126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.467353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.467543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.467769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.468461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.468655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.468847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] role.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.469039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.469230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] role.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.469448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.469634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.469814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.469994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.470179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.470179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.470538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.470538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.470889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.471964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.472967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473331 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473447 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.473907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.474969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.475902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.476931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.connection = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.477992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.478944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.479996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.480983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.connection = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.481162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.482843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.483023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.483261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.484119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.486329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.486329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.486329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:14:01.486329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 00:14:01.511945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.512245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.512460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.514146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.514377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.514940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.521407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.523564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.551992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.552195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.555782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.556018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.622731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.624179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.673836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.674088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.675134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.675404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.833863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.834088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.834318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.834547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.834765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.834982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.837432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.837672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.837897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.838120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.838338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.838702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.863877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.864017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.864101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.864237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.864343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.864481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.864577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.876355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.877192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.889292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.889502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.890958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.891100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.891215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.891295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.891402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.891485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:14:01.913016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.913220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:14:01.913563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.913655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.913762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:14:01.913919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:14:01.914038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.914264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.914345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.914464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:14:01.914554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.914739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.914820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.914999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.915519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.915779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.924457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.924741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.925013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.925299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.925493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.931324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.931324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.931324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.932268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.933281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.935191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.935481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.935771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.936055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.936217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.936355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.936599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.936775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.936858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.937979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.938166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.938245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.938546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.938862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.939096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.961687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.962731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.963157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.963442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.964195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.964870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.968637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.968924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.969213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.969995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.970367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.970560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.975772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.976078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.977088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.977358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.980059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.980369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.980660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.980944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.981218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.981482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:01.981859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.982069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.982354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.982637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.984013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.984013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.985124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.985594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.986725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:01.986725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.001172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.001461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.002226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.002518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.005392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.006392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.008525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.008824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.009632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.009929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.013189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.014183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.016446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.017641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.019538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.019856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.020144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.020454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.020736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.020867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.021119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.021204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.021448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.021802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.022054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.022325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.038000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.038306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.038579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.038910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.041933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.042237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.042508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.042771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.045857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.048523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.050092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.050379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.050518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.052062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.053065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.062754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.062754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.062754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.062754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.063193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.063193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.063193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.063193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.063193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.063193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.063820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.066087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.066391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.066677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.066960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.067237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.067517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.067817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.084533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.085294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.087734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.088318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.088700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.088954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.089068 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.089505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.089988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.092623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.094767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.096449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.096761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.097136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.104911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.106328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.109487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.110098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.110542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.111010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.111488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.114558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.118732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.122240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.122240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.122397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.122472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.122545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.122680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.125405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.128912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.129133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.129558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.130108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.130565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.130866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.131155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.131440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.131718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.131993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.132275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.132578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.132850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.133132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.135005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.137725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.144048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.144469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.144755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.145033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.155677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.156194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.156631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.166544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:14:02.166846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.167131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.167409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.167759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.168042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.168313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.168616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.168897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.169181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.169451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.169712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.169976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.170256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:14:02.176466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5582bd9629c0 pid: 88219 (default app) Aug 31 00:14:02.180443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82d9d-62e7-4f96-b03f-be56a67896e6 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:02.180731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82d9d-62e7-4f96-b03f-be56a67896e6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:02.181009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82d9d-62e7-4f96-b03f-be56a67896e6 None None] PATH_INFO: `/v3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:02.185299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Wed Aug 31 00:14:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:02.208568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5582bd9629c0 pid: 88218 (default app) Aug 31 00:14:10.076241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:10.076786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:10.076786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:10.078156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 2/2] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:10.085759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:10.086244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:10.086339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:10.157292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: INFO dbcounter [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] Registered counter for database keystone Aug 31 00:14:10.179268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG oslo_db.sqlalchemy.engines [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88218) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 00:14:10.185618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writer thread running {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 00:14:10.375083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG passlib.handlers.bcrypt [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88218) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 00:14:10.375694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG passlib.handlers.bcrypt [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88218) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 00:14:10.435536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:10.437864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:10.584576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1/3] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 00:14:11.079323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:11.150006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: INFO dbcounter [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] Registered counter for database keystone Aug 31 00:14:11.169156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG oslo_db.sqlalchemy.engines [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88219) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 00:14:11.176661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writer thread running {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 00:14:11.299518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ac76-8215-4dc1-9454-3113da451e71 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:11.304012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:11.304661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:11.304936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:11.305204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:11.314057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:11.366182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING keystone.common.rbac_enforcer.enforcer [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 00:14:11.366448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb035b-84e9-4d5d-be0e-8d7cfdaa7862 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:11.376778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 3/4] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 31 00:14:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:14:11.383314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:11.400383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a8d6f-c34c-44d7-b686-d6fd0a0096d0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:11.404935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:11.406703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:11.406976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:11.407247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] PATH_INFO: `/v3/projects/130b40029b174542ac4b0eacc3caf048` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:11.408742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] RBAC: Authorizing `identity:get_project(project_id=130b40029b174542ac4b0eacc3caf048)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:11.461296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 00:14:11.461635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca14075-891b-4b43-b382-0bc0b50e9889 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:11.466311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 2/5] 10.222.0.65 () {56 vars in 1207 bytes} [Wed Aug 31 00:14:11 2022] GET /identity/v3/projects/130b40029b174542ac4b0eacc3caf048 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:13.369757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dcf90-56fc-4139-9fee-e256625d0d29 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:13.370273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dcf90-56fc-4139-9fee-e256625d0d29 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:13.370358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dcf90-56fc-4139-9fee-e256625d0d29 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:13.375071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 4/6] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:13.380911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:13.381391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:13.381476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:13.401762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:13.403291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:13.411944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 3/7] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 00:14:13.687114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dcf90-56fc-4139-9fee-e256625d0d29 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:13.706478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dcf90-56fc-4139-9fee-e256625d0d29 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:13.713051 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13add467-945b-45b8-a0a0-6b21532c35d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:13.713802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13add467-945b-45b8-a0a0-6b21532c35d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:13.714267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13add467-945b-45b8-a0a0-6b21532c35d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:13.714717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13add467-945b-45b8-a0a0-6b21532c35d0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:13.727966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13add467-945b-45b8-a0a0-6b21532c35d0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:13.730611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13add467-945b-45b8-a0a0-6b21532c35d0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:13.740068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 5/8] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 31 00:14:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 00:14:13.746195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:13.755310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5969-d00b-415d-abc4-b14f7c7a1b0f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:13.759640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f9e26-ff84-4cb3-a449-a5b0fcdbd5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:13.760266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9e26-ff84-4cb3-a449-a5b0fcdbd5aa admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:13.760580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9e26-ff84-4cb3-a449-a5b0fcdbd5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:13.760850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9e26-ff84-4cb3-a449-a5b0fcdbd5aa admin admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:13.762160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f9e26-ff84-4cb3-a449-a5b0fcdbd5aa admin admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:13.764010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f9e26-ff84-4cb3-a449-a5b0fcdbd5aa admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:13.766218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 4/9] 10.222.0.65 () {56 vars in 1198 bytes} [Wed Aug 31 00:14:13 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:15.787847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574041-68f5-4b99-90b4-1bbe1bba79ee None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:15.788425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574041-68f5-4b99-90b4-1bbe1bba79ee None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:15.788425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574041-68f5-4b99-90b4-1bbe1bba79ee None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:15.789766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 6/10] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:15.795391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:15.795859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:15.795939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:15.816107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:15.817448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:15.866116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6653d-22a0-4d1c-93ab-40c676640550 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:15.866400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6653d-22a0-4d1c-93ab-40c676640550 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:15.866676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6653d-22a0-4d1c-93ab-40c676640550 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:15.868003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 7/11] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:15.873527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:15.873813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:15.874105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:15.900606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 5/12] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:15.917229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG passlib.handlers.bcrypt [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88219) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 00:14:15.917611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG passlib.handlers.bcrypt [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88219) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 00:14:15.967503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:15.969144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:16.072829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 8/13] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:16.173331 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.198186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.283156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f30fd1-43de-4d10-b7ee-371b18a5095a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.317279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.317925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.318209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.318496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.331677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:16.340518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:16.341366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:16.342569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 6/14] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:16.348095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.348240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.437572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.456013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491dac-b4a9-4ad4-b3cc-0497621a463c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.460520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e447eb-13ff-42fc-859d-754dadf3d73a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.462885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.493475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.494093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.494371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.494639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.502216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.503101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.503407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.503781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.504661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:16.504969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:16.506422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:16.519730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 9/15] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:16.525345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.573369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:16.574944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:16.634628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e15b-36af-4b2b-9813-f11d166a6c05 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.657772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.669523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.670378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.670652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.680751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.680751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:16.680751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:16.680751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 10/16] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:16.681252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 7/17] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 00:14:16.681352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.682281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.717749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.784252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbf24f-63bb-4c42-82b8-f4be7c1cfad1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.788198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38621ce4-f469-4d9e-9635-2c3ab6fd446d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.819551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.820480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.820763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.821030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.826121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.826748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.827029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.827484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.828824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:16.830285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:16.831518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:16.831844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 11/18] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:16.833238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:16.837797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:14:16.849278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:16.850387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:14:16.851164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:14:16.851164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:14:16.851164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:16.851164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 8/19] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:14:16.856211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:16.893917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.951999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aee86a-1673-4774-a5c9-1a477b517d33 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.953629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:16.963425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b8724-4fee-4f84-aa11-31736a5840a4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:16.988031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:16.988851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:16.989193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:16.989766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:16.990495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.domain.id=default, user.id=3b2336264fcc4528b59320d71b7da06f) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:16.990804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:16.992257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97710505-e525-444b-ac8f-474d9b00e502 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:16.999431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:17.000036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:17.000310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:17.000607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:17.001244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:17.001374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 12/20] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/role_assignments?user.id=3b2336264fcc4528b59320d71b7da06f&scope.domain.id=default&role.id=ac2134ae61074ce6b751deadce042c42 => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:17.001542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:17.002935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7eb23-82e3-48c5-b39d-020f4592db47 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:17.022499 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 9/21] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:19.083476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804291-ed6a-4825-8721-82cf1dd48844 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.084526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804291-ed6a-4825-8721-82cf1dd48844 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.085074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804291-ed6a-4825-8721-82cf1dd48844 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.087101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 13/22] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:19.092618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ada1fb-ab18-44aa-a998-afd00a407739 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.094101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ada1fb-ab18-44aa-a998-afd00a407739 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.094575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ada1fb-ab18-44aa-a998-afd00a407739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.121584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-66ada1fb-ab18-44aa-a998-afd00a407739 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:19.123165 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-66ada1fb-ab18-44aa-a998-afd00a407739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:19.218948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 10/23] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:19.413330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd027fd-b27e-4c62-9360-85e8c4378442 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.413976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd027fd-b27e-4c62-9360-85e8c4378442 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.414496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd027fd-b27e-4c62-9360-85e8c4378442 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.416383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 14/24] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:19.423100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf727f-cb91-4046-a069-c6fecc582252 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.423691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf727f-cb91-4046-a069-c6fecc582252 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.424167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf727f-cb91-4046-a069-c6fecc582252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.453513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-25bf727f-cb91-4046-a069-c6fecc582252 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:19.455088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-25bf727f-cb91-4046-a069-c6fecc582252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:19.488608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:19.559070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 11/25] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:19.605669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f4638-4974-47db-89b5-6c71663746fa None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.606324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f4638-4974-47db-89b5-6c71663746fa None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.606801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f4638-4974-47db-89b5-6c71663746fa None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.609175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 15/26] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:19.614877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.615442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.615942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.642915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:19.644449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f4638-4974-47db-89b5-6c71663746fa None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:19.654898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:19.784444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 12/27] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:19.830003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f4638-4974-47db-89b5-6c71663746fa None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:19.882938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8654c-397e-4191-8d41-5f0c4d114cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:19.883835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8654c-397e-4191-8d41-5f0c4d114cec None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:19.884262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8654c-397e-4191-8d41-5f0c4d114cec None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:19.884701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8654c-397e-4191-8d41-5f0c4d114cec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:19.906876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8654c-397e-4191-8d41-5f0c4d114cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-67a8654c-397e-4191-8d41-5f0c4d114cec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:19.915964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:19.917029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:19.918038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:19.920020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 16/28] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:19.926022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.030829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73a28-b9da-4984-ae5b-e471854bdf24 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.031434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73a28-b9da-4984-ae5b-e471854bdf24 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.031855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73a28-b9da-4984-ae5b-e471854bdf24 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.033444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 17/29] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:20.036708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:20.039379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.039664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.039938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.067130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:20.068367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:20.082559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36a2c7-2b46-44b3-9200-6e41bb6e3963 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:20.096785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:20.151127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 18/30] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:20.152561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.172487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:20.174003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.175875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.176401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.177307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:20.178318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:20.180293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:20.209175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 13/31] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:20.215239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.342090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06014e8-f2c0-41f5-8638-a50d0b01e29f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:20.366696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fede3-2005-4da5-b16b-3fe8293352f0 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:20.387592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:20.390757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.392315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.392918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.396115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:20.400216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:20.406717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 14/32] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:20 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:20.407800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.442136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:20.443253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.445148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.445633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.446894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:20.496808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:20.498251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:14:20.531768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:20.532699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:14:20.533544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:14:20.533544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:14:20.533544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:20.536076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 19/33] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:20.537262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.554339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:20.575793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b4e6d-532d-4f1a-8142-9ed95321c06d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:20.616962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:20.617668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.617950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.618524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.619535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:20.621282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:20.675953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40c707-39ec-448f-8733-3bb1c7bdb7e6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:20.729783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:20.730480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.730763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.731134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.732566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:20.734614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:20.736546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 20/34] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:20.738150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.849090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:20.853512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 15/35] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 00:14:20 2022] POST /identity/v3/roles => generated 225 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:20.854867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:20.900529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84979f-5d18-48b7-a6c0-7d30595f259d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:20.986453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f4309-44bb-4c2f-b3d6-4fbecbddbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:20.987463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4309-44bb-4c2f-b3d6-4fbecbddbdef None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.987943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4309-44bb-4c2f-b3d6-4fbecbddbdef None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.988440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4309-44bb-4c2f-b3d6-4fbecbddbdef None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.989382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f4309-44bb-4c2f-b3d6-4fbecbddbdef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:20.991245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f4309-44bb-4c2f-b3d6-4fbecbddbdef None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:21.031930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:21.064295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251960d0-56bc-4bb3-a21f-3a578f23841c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:21.107057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:21.107706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.107991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.108899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.110006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:21.110419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:21.112096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:21.129448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 16/36] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:21.130157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:21.213572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 21/37] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 00:14:20 2022] POST /identity/v3/roles => generated 219 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:21.215477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4b95-77c3-4d2b-ac17-7069806d9434 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.215872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4b95-77c3-4d2b-ac17-7069806d9434 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.216276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4b95-77c3-4d2b-ac17-7069806d9434 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.222009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 22/38] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:21.222009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca4b95-77c3-4d2b-ac17-7069806d9434 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:21.275493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcd0f6-a32e-4f2f-a4c8-c0017cc6cb8d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:21.376232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca4b95-77c3-4d2b-ac17-7069806d9434 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:21.428837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d27a0-c710-4975-a000-05f10583a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:21.429537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d27a0-c710-4975-a000-05f10583a66a None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.429818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d27a0-c710-4975-a000-05f10583a66a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.430096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d27a0-c710-4975-a000-05f10583a66a None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.430936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d27a0-c710-4975-a000-05f10583a66a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:21.432551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d27a0-c710-4975-a000-05f10583a66a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:21.434992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068e360-8718-4e22-9959-dea0d010222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:21.435667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e360-8718-4e22-9959-dea0d010222d None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.435952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e360-8718-4e22-9959-dea0d010222d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.436235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068e360-8718-4e22-9959-dea0d010222d None admin] PATH_INFO: `/v3/domains/default/users/3b2336264fcc4528b59320d71b7da06f/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.439345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068e360-8718-4e22-9959-dea0d010222d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3b2336264fcc4528b59320d71b7da06f, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:21.440301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:21.440928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068e360-8718-4e22-9959-dea0d010222d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:21.443602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:21.535238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 17/39] 10.222.0.65 () {58 vars in 1361 bytes} [Wed Aug 31 00:14:21 2022] PUT /identity/v3/domains/default/users/3b2336264fcc4528b59320d71b7da06f/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:14:21.536693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b8cc7-6dbc-40f1-963a-38e11ebb6707 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.536977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b8cc7-6dbc-40f1-963a-38e11ebb6707 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.537253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b8cc7-6dbc-40f1-963a-38e11ebb6707 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.538606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 18/40] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:21.539782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.540091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.540437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.560290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 23/41] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:14:21 2022] POST /identity/v3/projects => generated 301 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:21.564764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:21.566608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:21.574012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.574365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.574575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.618462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:21.624993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:21.678396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 19/42] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:21.731300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 24/43] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:21.954608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:22.018536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:22.068419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e46bc-8c4f-44d1-9fb7-f73e2a8a0a9f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:22.096110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:22.107983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4171f-7869-4b3a-aac3-70ab2e8fdcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:22.109161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4171f-7869-4b3a-aac3-70ab2e8fdcc1 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:22.109444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4171f-7869-4b3a-aac3-70ab2e8fdcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:22.109715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4171f-7869-4b3a-aac3-70ab2e8fdcc1 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:22.110382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4171f-7869-4b3a-aac3-70ab2e8fdcc1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:22.111813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4171f-7869-4b3a-aac3-70ab2e8fdcc1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:22.138475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219e3e1-ec52-42f4-a977-f578e3881580 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:22.156356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:22.174128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:22.174769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:22.175050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:22.175325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:22.176463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:22.177925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:22.179496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 25/44] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:22.184869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:22.212075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 20/45] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 00:14:21 2022] POST /identity/v3/roles => generated 223 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:22.308713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225139-5f1c-4b98-90d1-0a1502c3e268 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:22.356836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949de6c4-0b2f-478b-ba0f-1c55b1c8b9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:22.357461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de6c4-0b2f-478b-ba0f-1c55b1c8b9d6 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:22.357745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de6c4-0b2f-478b-ba0f-1c55b1c8b9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:22.358024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de6c4-0b2f-478b-ba0f-1c55b1c8b9d6 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:22.358745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949de6c4-0b2f-478b-ba0f-1c55b1c8b9d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:22.360202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949de6c4-0b2f-478b-ba0f-1c55b1c8b9d6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:22.547870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 26/46] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:14:22 2022] POST /identity/v3/projects => generated 312 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:14:23.546547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e8d6-1c69-4992-87d5-beb89c1785a7 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:23.547047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e8d6-1c69-4992-87d5-beb89c1785a7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:23.547222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e8d6-1c69-4992-87d5-beb89c1785a7 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:23.548656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 21/47] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:23.554110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:23.554531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:23.554671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:23.574255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:23.575487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:23.607092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:23.647998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 27/48] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:23.917607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e8d6-1c69-4992-87d5-beb89c1785a7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.021967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e8d6-1c69-4992-87d5-beb89c1785a7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:24.057700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:24.058323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.058603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.058873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.070144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:24.074985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:24.075822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:24.076619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 22/49] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:24.082534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.188155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47265c-d0b1-4252-bf00-71a63c6b9de8 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:24.224891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:24.225503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.225778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.226046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.226710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:24.227013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:24.228441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:24.238136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 28/50] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:24.244303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.292146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:24.353218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae54629-e3c5-4a15-9b03-b9a69f5e75f1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:24.388122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee46236-cd06-4e49-b9d8-a43553029623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:24.388788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee46236-cd06-4e49-b9d8-a43553029623 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.389074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee46236-cd06-4e49-b9d8-a43553029623 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.389878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee46236-cd06-4e49-b9d8-a43553029623 None admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.391080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee46236-cd06-4e49-b9d8-a43553029623 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:24.392594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee46236-cd06-4e49-b9d8-a43553029623 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:24.394207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 23/51] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:24 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:24.399864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.432248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:24.481711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d847-3776-4b26-92c4-9334f01d38be None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.481965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d847-3776-4b26-92c4-9334f01d38be None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.482585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30d847-3776-4b26-92c4-9334f01d38be None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.483596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 24/52] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:24.490123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.490444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.490721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.510900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:24.511165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a9b4-c051-44a3-ad4d-627f14a8c45c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:24.512091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:24.548674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:24.549315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.549598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.549870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.551000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:24.552523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:24.554095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 29/53] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:24.559836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.588547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 25/54] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:24.669528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:24.674050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3decb-9f4d-4030-9a35-6de708687837 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:24.820248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:24.820886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.821263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.821500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.822182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.domain.id=default, user.id=3b2336264fcc4528b59320d71b7da06f) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:24.822490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:24.824652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:24.831853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 30/55] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 31 00:14:24 2022] GET /identity/v3/role_assignments?user.id=3b2336264fcc4528b59320d71b7da06f&scope.domain.id=default&role.id=ac2134ae61074ce6b751deadce042c42 => generated 530 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:24.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.951023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:25.023997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c110dc-2f86-4b52-b13e-d9e01d64894a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:25.064844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26610e34-aaac-4d37-bbb3-f91f81b0c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:25.065495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610e34-aaac-4d37-bbb3-f91f81b0c323 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:25.065779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610e34-aaac-4d37-bbb3-f91f81b0c323 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:25.066054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610e34-aaac-4d37-bbb3-f91f81b0c323 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:25.067198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610e34-aaac-4d37-bbb3-f91f81b0c323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:25.068726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610e34-aaac-4d37-bbb3-f91f81b0c323 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:25.070614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 26/56] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:25.076238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:25.188398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813cc8-509a-407e-802a-c72c5cab4181 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:25.225352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9e82b-8e29-4f72-9b50-06b47f3675e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:25.225967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e82b-8e29-4f72-9b50-06b47f3675e3 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:25.226246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e82b-8e29-4f72-9b50-06b47f3675e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:25.226516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e82b-8e29-4f72-9b50-06b47f3675e3 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:25.227172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9e82b-8e29-4f72-9b50-06b47f3675e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:25.228633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9e82b-8e29-4f72-9b50-06b47f3675e3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:25.326524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 31/57] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 31 00:14:25 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:27.271048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afa8c9-f49b-4187-9d52-9a097ab60d7d None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.271601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afa8c9-f49b-4187-9d52-9a097ab60d7d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.271601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afa8c9-f49b-4187-9d52-9a097ab60d7d None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.272956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 27/58] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:27.278336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.278757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.278863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.297692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:27.298916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:27.308141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:27.370046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 32/59] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:27.644652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afa8c9-f49b-4187-9d52-9a097ab60d7d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:27.746926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afa8c9-f49b-4187-9d52-9a097ab60d7d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:27.757901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:27.781954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011572b9-bbdd-464e-b8dc-a81feb8e7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:27.782577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011572b9-bbdd-464e-b8dc-a81feb8e7484 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.782863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011572b9-bbdd-464e-b8dc-a81feb8e7484 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.783140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011572b9-bbdd-464e-b8dc-a81feb8e7484 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.784265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011572b9-bbdd-464e-b8dc-a81feb8e7484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:27.785751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011572b9-bbdd-464e-b8dc-a81feb8e7484 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:27.787611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 28/60] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:27.793001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:27.892338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827106b-6dce-4080-a793-29318cbf7d0c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:27.926556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d8e8b-7f6f-4edc-9ab2-c32bd5a8313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:27.927090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d8e8b-7f6f-4edc-9ab2-c32bd5a8313e None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.927371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d8e8b-7f6f-4edc-9ab2-c32bd5a8313e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.927636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d8e8b-7f6f-4edc-9ab2-c32bd5a8313e None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.928271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d8e8b-7f6f-4edc-9ab2-c32bd5a8313e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:27.929745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d8e8b-7f6f-4edc-9ab2-c32bd5a8313e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:28.027247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:28.032448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 33/61] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:14:27 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:31.244059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9beae-05e7-4536-8b70-0f5299e56b72 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.244059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9beae-05e7-4536-8b70-0f5299e56b72 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.244059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9beae-05e7-4536-8b70-0f5299e56b72 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.244059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 29/62] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:31.249799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128ecf-5d71-4cd1-ba94-476f96bf0f94 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.250673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128ecf-5d71-4cd1-ba94-476f96bf0f94 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.251124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128ecf-5d71-4cd1-ba94-476f96bf0f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.281607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-40128ecf-5d71-4cd1-ba94-476f96bf0f94 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:31.283409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-40128ecf-5d71-4cd1-ba94-476f96bf0f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:31.379015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 34/63] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:31.387396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942b01-326a-445e-aa1f-d2427da90b5f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.388184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942b01-326a-445e-aa1f-d2427da90b5f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.388706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942b01-326a-445e-aa1f-d2427da90b5f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.390262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 30/64] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:31.399815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7c82c-2029-4ff9-a4f9-7a118785942a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.400097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7c82c-2029-4ff9-a4f9-7a118785942a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.400405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7c82c-2029-4ff9-a4f9-7a118785942a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.425237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-54a7c82c-2029-4ff9-a4f9-7a118785942a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:31.426491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-54a7c82c-2029-4ff9-a4f9-7a118785942a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:31.526544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 35/65] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:31.662084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb80f0d-0d18-45fd-a214-842a893b2e79 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.662769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb80f0d-0d18-45fd-a214-842a893b2e79 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.663269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb80f0d-0d18-45fd-a214-842a893b2e79 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.664871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 31/66] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:31.671251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40333-418c-4e78-8a2e-398142f0ee41 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.671539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40333-418c-4e78-8a2e-398142f0ee41 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.671817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a40333-418c-4e78-8a2e-398142f0ee41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.699509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a8a40333-418c-4e78-8a2e-398142f0ee41 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:31.699916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb80f0d-0d18-45fd-a214-842a893b2e79 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:31.700770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a8a40333-418c-4e78-8a2e-398142f0ee41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:31.800789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:31.814373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 36/67] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:31.865693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d92c-a1d4-4e27-9ed0-e023444de23f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.865977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d92c-a1d4-4e27-9ed0-e023444de23f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.866518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d92c-a1d4-4e27-9ed0-e023444de23f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.868084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 37/68] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:31.881200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb3999-467d-4349-ac82-076d8e7d29c2 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.881796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb3999-467d-4349-ac82-076d8e7d29c2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.882278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb3999-467d-4349-ac82-076d8e7d29c2 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.883996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 38/69] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:31.885458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7c029-bb3b-4daa-93a6-727eeed7b3fe None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.885732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7c029-bb3b-4daa-93a6-727eeed7b3fe None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.886213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7c029-bb3b-4daa-93a6-727eeed7b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.888711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb80f0d-0d18-45fd-a214-842a893b2e79 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:31.908116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e1d7c029-bb3b-4daa-93a6-727eeed7b3fe None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:31.909361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e1d7c029-bb3b-4daa-93a6-727eeed7b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:31.931846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:31.932517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.932805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.933074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.946710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:31.951646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:31.953547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:31.955163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 32/70] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:31.957181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.004752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 39/71] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:32.006236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c49219-688e-4252-967b-361521b9f62d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.006530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c49219-688e-4252-967b-361521b9f62d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.006808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c49219-688e-4252-967b-361521b9f62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.010014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.031601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f2c49219-688e-4252-967b-361521b9f62d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:32.034197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f2c49219-688e-4252-967b-361521b9f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:32.081407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03737bbc-0e99-42cd-b859-686edbaa8030 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.119667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 40/72] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:32.120728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c49219-688e-4252-967b-361521b9f62d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.130682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.132137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.132742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.133228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.146223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:32.151543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:32.152418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:32.153179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:32.156486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 33/73] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:32.158857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.188149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.247815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c49219-688e-4252-967b-361521b9f62d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.275575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.279913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ae5b1-f7d1-4b3f-9011-f97bededd0ad None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.286790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.288463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.288463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.288463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.288781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:32.289406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.290635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.304666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 41/74] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:32.304666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.326472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.327276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.327566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.327828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.341186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:32.350020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:32.356371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 34/75] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:32.357443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.465362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ac224-a4f9-47ab-af15-3d8d8eaddf0a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.479502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.488593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a633d-132e-4c62-9b1a-b157de490e64 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.504106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.504748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.505020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.505292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.505950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:32.506250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.507620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.517714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 42/76] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:32.518579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.530555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.531246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.531486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.531755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.533663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.535133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.536728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 35/77] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:32.537483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.566059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.624903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbeff2-01bb-4d49-ab81-6365497a1d98 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.650219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64337-467e-47fb-af24-749f4a0a19e5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.663601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.664275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.664640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.664918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.666152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.667692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.669317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 43/78] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:32.670097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.691169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.691797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.692082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.692378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.693057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:32.693365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.695114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.703642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 36/79] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:32.703642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.719281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.791719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798848f-d2f7-4239-bb7f-7e7b13d3a7b3 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.805453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.817174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb038d-c1b8-479c-9d3e-75e867d71c5d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.829250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.830121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.830429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.830703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.841702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:32.847455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:32.848442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:32.849218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 44/80] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:32.849847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.857704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.858352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.858640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.858917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.873390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.874865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.876663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 37/81] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:32.877404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.956840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff24a4-8ebe-4ba1-b786-da10917be7a5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.958406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.981179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f789b-098a-4969-81c5-653d2aba313a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.992089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.992734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.993024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.993293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.993949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.995401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.018697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.019311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.019594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.019871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.036616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.079837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 45/82] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:14:32 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:33.080826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.093364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.095702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.221734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 38/83] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:33.222647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.225431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628b8d0-ee4a-4174-8f08-0063c50bd528 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.271147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.271742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.271742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.272120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.273168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.274636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.276270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 46/84] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:33.277066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.285443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.311134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.347624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19a88-e419-4395-bbf5-c976e76e6d22 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.395520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.396148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.396466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.396745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.397428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:33.397730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.399141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.412074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 39/85] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:33.414513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.452719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941bec-fabd-4433-9ee0-3ee41de095bb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.512630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.513266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.513546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.514318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.540657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b4611-cae9-4046-86ea-68e55bbbc85e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.551980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.581101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.587269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.587269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.587269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.587269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.587893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.591300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 40/86] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:33.592818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.615056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.617089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.655219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.744102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300219-833e-4bcb-97db-11cffdb6a7f9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.758039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 47/87] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:33.758979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.789538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.790378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.790660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.790933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.853447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.878308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.879795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.897264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14184f8f-fd8f-462e-830d-a71b2ae996c7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.939931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.950730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.951393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.956773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.956773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.956773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.956773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:33.957480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 48/88] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:33.958792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.009100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 41/89] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:34.011038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.069929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4817046-bc81-406b-98e3-e51681c12a62 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:34.107024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:34.107410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:34.107827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:34.108477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:34.108477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:34.109316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=3b2336264fcc4528b59320d71b7da06f) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:34.109765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:34.111475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:34.118160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 49/90] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/role_assignments?user.id=3b2336264fcc4528b59320d71b7da06f&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:34.125313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.137809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992dca0-6b91-438a-945b-82bc3c6d1755 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:34.182391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:34.183593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:34.185658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:34.186248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:34.189276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:34.192181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:34.194451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 42/91] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:34 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:34.195549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.243293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:34.321368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290750d-2d24-46c0-9455-63e3b4acb3ab None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:34.340119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea83e36-4f1a-401d-b27b-dd64b2a20dc4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:34.362358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:34.363129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:34.363310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:34.363561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:34.391485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:34.392587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:34.393094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:34.393584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:34.395220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:34.397167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:34.399151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 43/92] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:34 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:34.400186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.422136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:34.468676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:34.471291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:34.581826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2ccfd-23a6-4625-b007-f5d918b91f16 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:34.604731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:34.667398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:34.668059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:34.668355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:34.668636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:34.669409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=4e79330e00e34ea08d7152d870440c86) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:34.669934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:34.671839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:34.686419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 50/93] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:34.687442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.692903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 44/94] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:34 2022] GET /identity/v3/role_assignments?user.id=4e79330e00e34ea08d7152d870440c86&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:34.693863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:34.884902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:34.933858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00777aab-a5fe-4830-9e73-c88710487127 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:34.957841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d38dae-8c89-47a1-ac71-2b1cee19484d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:35.022202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:35.022820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.023099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.023372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.024110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] RBAC: Adding query filter params (role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=4e79330e00e34ea08d7152d870440c86) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:35.024694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:35.026557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:35.034509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 51/95] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:34 2022] GET /identity/v3/role_assignments?user.id=4e79330e00e34ea08d7152d870440c86&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:35.101679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29cf16-c644-417b-8dc2-2b838f577aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:35.102775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29cf16-c644-417b-8dc2-2b838f577aaa None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.103286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29cf16-c644-417b-8dc2-2b838f577aaa None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.103741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29cf16-c644-417b-8dc2-2b838f577aaa None admin] PATH_INFO: `/v3/projects/4ff18af8479444da8d5e1f6d00df6351` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.105802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29cf16-c644-417b-8dc2-2b838f577aaa None admin] RBAC: Authorizing `identity:get_project(project_id=4ff18af8479444da8d5e1f6d00df6351)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:35.108302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29cf16-c644-417b-8dc2-2b838f577aaa None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:35.111520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 45/96] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:34 2022] GET /identity/v3/projects/4ff18af8479444da8d5e1f6d00df6351 => generated 312 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:35.117669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:35.155778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:35.258682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0003d-d476-4979-b684-125eba258b0e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:35.297146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:35.298467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.298807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.299083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.299841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=4ff18af8479444da8d5e1f6d00df6351, user.id=4e79330e00e34ea08d7152d870440c86) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:35.300152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:35.301652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9e56-8052-49a5-bc1b-f56678e27824 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:35.308078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 52/97] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:35 2022] GET /identity/v3/role_assignments?user.id=4e79330e00e34ea08d7152d870440c86&scope.project.id=4ff18af8479444da8d5e1f6d00df6351&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:35.563561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c38e-85a5-4623-9aa3-1a19641fc9e7 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.564704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c38e-85a5-4623-9aa3-1a19641fc9e7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.565275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9c38e-85a5-4623-9aa3-1a19641fc9e7 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.567502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 46/98] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:35.576796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e0309-e685-42ad-b7b5-62e75636992d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.578180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e0309-e685-42ad-b7b5-62e75636992d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.579517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e0309-e685-42ad-b7b5-62e75636992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.613809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-922e0309-e685-42ad-b7b5-62e75636992d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:35.619431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-922e0309-e685-42ad-b7b5-62e75636992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:35.764682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 53/99] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:35.893888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64ed4d-54c2-4e9e-afbd-87a124810b91 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.894573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64ed4d-54c2-4e9e-afbd-87a124810b91 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.895048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64ed4d-54c2-4e9e-afbd-87a124810b91 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.896652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 47/100] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:35.902494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.903183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.903703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.933951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:35.935504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:35.954436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:36.028499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 54/101] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:36.307150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64ed4d-54c2-4e9e-afbd-87a124810b91 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:36.445342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64ed4d-54c2-4e9e-afbd-87a124810b91 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:36.497260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:36.497890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:36.498270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:36.498548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:36.511677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:36.518855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:36.519945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:36.520736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 48/102] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:14:36.526723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:36.580180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:36.584813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:36.760387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d895485-d3b9-4965-8019-ae74dcc20e2d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:36.886518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dc060-b407-4789-be17-242bbcfeccaf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:36.894098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:36.894098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:36.894098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:36.894636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:36.895058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:36.895335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:36.896724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:36.906888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 55/103] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:36.913809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:36.935881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:36.973622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:36.975020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:36.975740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:36.976542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:36.978983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:36.982030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:36.986249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 49/104] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:37.004571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:37.128909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1abe5-aa6e-4ae4-a984-3d28f78c2d4a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:37.200187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312d1c-ca49-4708-ac34-3cfc998d574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:37.200889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312d1c-ca49-4708-ac34-3cfc998d574b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.201168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312d1c-ca49-4708-ac34-3cfc998d574b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.201446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312d1c-ca49-4708-ac34-3cfc998d574b None admin] PATH_INFO: `/v3/users/a0909593eafe40ac85627b139c0cfc8d` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.217910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:37.221725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31312d1c-ca49-4708-ac34-3cfc998d574b None admin] RBAC: Authorizing `identity:get_user(user_id=a0909593eafe40ac85627b139c0cfc8d)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:37.223339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31312d1c-ca49-4708-ac34-3cfc998d574b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:37.225607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 56/105] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:36 2022] GET /identity/v3/users/a0909593eafe40ac85627b139c0cfc8d => generated 289 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:37.229966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ddc2-8084-41e2-8ac7-202b74a14ad1 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.229966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ddc2-8084-41e2-8ac7-202b74a14ad1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.229966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ddc2-8084-41e2-8ac7-202b74a14ad1 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.229966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 57/106] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:37.237055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.237349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.237930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.259532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:37.260912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:37.327220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc83de-a625-4e95-beab-faf5f3696d19 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:37.392809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:37.404575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 58/107] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:37.405436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:37.415618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f20f-03e4-4323-bc09-591658b457a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:37.415618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f20f-03e4-4323-bc09-591658b457a2 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.415618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f20f-03e4-4323-bc09-591658b457a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.415618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f20f-03e4-4323-bc09-591658b457a2 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.416227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180f20f-03e4-4323-bc09-591658b457a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:37.418341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180f20f-03e4-4323-bc09-591658b457a2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:37.537395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 50/108] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:14:37 2022] POST /identity/v3/projects => generated 302 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:37.615610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe16-e7ef-4719-bdfb-e0cfec82c34d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:37.668081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:37.668940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.669498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.669934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.773391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:37.784239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:37.786106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:37.935521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47766d71-0f99-4c83-afe0-dcc5be4ebf00 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.936287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47766d71-0f99-4c83-afe0-dcc5be4ebf00 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.936287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47766d71-0f99-4c83-afe0-dcc5be4ebf00 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.937575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 51/109] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:37.944800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 59/110] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:37.946204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:37.946697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:37.947169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:37.956348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:37.979775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:37.981355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:38.070675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 52/111] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:38.071494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:38.097364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:38.101993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185531e-7ce9-4eff-85ba-1a1528e3aea3 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:38.147163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:38.147807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.148239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.148664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.149983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:38.152032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:38.154338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 60/112] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:37 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:38.155595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:38.161226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:38.208368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20714990-4080-4c75-a57c-19bf659e1340 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:38.253129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:38.254227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.254617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.254990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.272823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:38.279429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:38.280298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:38.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:38.282379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 53/113] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:38.289057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:38.293467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1190a55-94b6-4c1d-ad65-689ab6df1537 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:38.349984 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:38.350552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.351098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.351374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.352552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] RBAC: Adding query filter params (role.id=de69739cb03e44e4bad4acc69a2f8e72, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=a0909593eafe40ac85627b139c0cfc8d) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:38.352864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:38.354322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:38.363476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 61/114] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:38 2022] GET /identity/v3/role_assignments?user.id=a0909593eafe40ac85627b139c0cfc8d&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=de69739cb03e44e4bad4acc69a2f8e72 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:38.491942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca1e5-9f4b-439c-b11b-5d0d10bcee1a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:38.530530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:38.572988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:38.574189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.574890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.575685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.576405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:38.576715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:38.578179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:38.591574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 54/115] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:38.598621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:38.629874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:38.668753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:38.789573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0b06e-b4a0-41db-a01b-58768ddb703f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:38.821065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d84a-a3d3-4980-a631-9f67e2e8d5cb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:38.839506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643ee67-e339-494b-8ad6-23f8ca3a7510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:38.840582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643ee67-e339-494b-8ad6-23f8ca3a7510 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.841046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643ee67-e339-494b-8ad6-23f8ca3a7510 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.841486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643ee67-e339-494b-8ad6-23f8ca3a7510 None admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.843298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643ee67-e339-494b-8ad6-23f8ca3a7510 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:38.845619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643ee67-e339-494b-8ad6-23f8ca3a7510 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:38.847525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 62/116] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:38 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:38.848800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce60a5-7254-4a05-9682-4c107aef5a06 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.849083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce60a5-7254-4a05-9682-4c107aef5a06 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.849360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce60a5-7254-4a05-9682-4c107aef5a06 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.850638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 63/117] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:38.851833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fd011-a9b1-4f8b-80a9-bf52978c1638 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.852254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fd011-a9b1-4f8b-80a9-bf52978c1638 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.852735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fd011-a9b1-4f8b-80a9-bf52978c1638 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.854034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 64/118] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:38.859569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fd011-a9b1-4f8b-80a9-bf52978c1638 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:38.874648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4010c09-1757-41a0-bede-1ff130113cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:38.876717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4010c09-1757-41a0-bede-1ff130113cf2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.877190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4010c09-1757-41a0-bede-1ff130113cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.877485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4010c09-1757-41a0-bede-1ff130113cf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.897164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4010c09-1757-41a0-bede-1ff130113cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e4010c09-1757-41a0-bede-1ff130113cf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:38.909188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:38.910086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:38.910835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:38.911935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 55/119] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:38.913206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086353af-2385-4d47-81e8-3befb59b2a10 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.913496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086353af-2385-4d47-81e8-3befb59b2a10 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.915221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086353af-2385-4d47-81e8-3befb59b2a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.940560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-086353af-2385-4d47-81e8-3befb59b2a10 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:38.943213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-086353af-2385-4d47-81e8-3befb59b2a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:38.977124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:39.008632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:39.018700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fd011-a9b1-4f8b-80a9-bf52978c1638 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.027792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 56/120] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:39.029423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.029780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.030124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.060107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:39.062090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:39.068092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.069324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.069627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.069901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.071389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:39.073012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:39.075156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 65/121] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:38 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:39.076355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.187066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 57/122] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:39.187828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.302595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a22e7a-5837-421b-a5d8-2dea1fb0686e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.390508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.391735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.392301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.392822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.393698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:39.394198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:39.395785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:39.416092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 66/123] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:39.422085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.423424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:39.448212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ffc5-4c78-4748-8f37-fabe5e465f27 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.454954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:39.554139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebc5cf-1e5b-485b-916b-5669518f159e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.576703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.577616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.577900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.578178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/3b2336264fcc4528b59320d71b7da06f/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.585985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7fe77a62546939bec3adf10e3790f, user_id=3b2336264fcc4528b59320d71b7da06f, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:39.587770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:39.610158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.610805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.611090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.611365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.612920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:39.614562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:39.616164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 67/124] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:39.617131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.665307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 58/125] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:39 2022] PUT /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/3b2336264fcc4528b59320d71b7da06f/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:14:39.672870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.759972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:39.762715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f3536-52ea-4cca-8662-688838ef0c97 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.820831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.821589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.821777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.824341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.853388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:39.863207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:39.864143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:39.864951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:39.877520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 68/126] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:39.878767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.882517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c036-22a6-4517-8557-54aebb73d34f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.926224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.926891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.927202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.927440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.928774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:39.930292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:39.932576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 59/127] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:39.933470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.967803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:40.078443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f9bf6-590a-44af-8f74-fdebc6b8726d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.084201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972aeb4-ff47-48c9-8766-37fd19aad6e4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.144566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.144925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.145718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.145904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.146215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.146380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.146660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.146823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.148051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:40.148580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:40.151188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:40.169826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 60/128] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:40.171951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:40.186367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:40.193212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:40.195397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 69/129] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:40.197836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:40.225335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:40.362386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:40.378850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc6c9b-a938-461a-bc0e-6ee54a2e3ece None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.421622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.424043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.424413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.424695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/4e79330e00e34ea08d7152d870440c86/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.426908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7fe77a62546939bec3adf10e3790f, user_id=4e79330e00e34ea08d7152d870440c86, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:40.428398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:40.466068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27b7ee-8928-4576-9416-885c876fc7fe None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.506060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.506753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.507035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.507310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.508506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:40.515851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:40.515851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 70/130] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:40 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:40.515851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:40.548932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 61/131] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:40 2022] PUT /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/4e79330e00e34ea08d7152d870440c86/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:14:40.549770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:40.666116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:40.685874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79746a0e-ef4c-4885-b5c0-d33183f86b69 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.725741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.726372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.726643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.726920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.727583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:40.727887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:40.729402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:40.739435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 71/132] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:14:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:40.745244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:40.798112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc640b-68ce-4b13-9a25-521d1b957e17 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.863722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2798a60-6693-4ccd-a580-b66afc91f4d8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:40.905973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.907999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.910471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.912010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.917391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:40.919090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:40.922105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 72/133] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:40 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:40.928825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:40.947972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:40.949102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:40.949607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:40.950085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] PATH_INFO: `/v3/projects/4ff18af8479444da8d5e1f6d00df6351` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:40.957793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff18af8479444da8d5e1f6d00df6351)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:40.960782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:40.968512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 62/134] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:40 2022] GET /identity/v3/projects/4ff18af8479444da8d5e1f6d00df6351 => generated 312 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:40.969744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:40.974535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:41.051020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:41.064701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58587bf2-559b-4933-a080-751a0904e0b7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:41.110504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:41.111170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.111450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.111722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.113011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:41.114850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:41.116616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 73/135] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:40 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:41.123300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:41.235407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:41.254759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff48929-2e4d-42c6-a8ac-4000561b95d3 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:41.294551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa6f04-8f09-4ac0-91b0-58d1b36898fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:41.295312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa6f04-8f09-4ac0-91b0-58d1b36898fe None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.295601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa6f04-8f09-4ac0-91b0-58d1b36898fe None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.296123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa6f04-8f09-4ac0-91b0-58d1b36898fe None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/4e79330e00e34ea08d7152d870440c86/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.299592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa6f04-8f09-4ac0-91b0-58d1b36898fe None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7fe77a62546939bec3adf10e3790f, user_id=4e79330e00e34ea08d7152d870440c86, role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:41.301143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa6f04-8f09-4ac0-91b0-58d1b36898fe None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:41.348947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d8358-cbf3-41aa-a22d-7debfd16b724 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:41.380879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 74/136] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:41 2022] PUT /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/4e79330e00e34ea08d7152d870440c86/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:14:41.408832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28787599-30f8-484c-891f-486113429ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:41.409588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28787599-30f8-484c-891f-486113429ca8 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.409871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28787599-30f8-484c-891f-486113429ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.410150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28787599-30f8-484c-891f-486113429ca8 None admin] PATH_INFO: `/v3/projects/4ff18af8479444da8d5e1f6d00df6351/users/4e79330e00e34ea08d7152d870440c86/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.412353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28787599-30f8-484c-891f-486113429ca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff18af8479444da8d5e1f6d00df6351, user_id=4e79330e00e34ea08d7152d870440c86, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:41.414293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28787599-30f8-484c-891f-486113429ca8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:41.455377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2770c5-26be-4fb1-a772-57913755df1f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.455805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2770c5-26be-4fb1-a772-57913755df1f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.456251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2770c5-26be-4fb1-a772-57913755df1f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.458814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 75/137] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:41.468414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99627a-8570-4cc5-a57a-e9898f0733c7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.468895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99627a-8570-4cc5-a57a-e9898f0733c7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.469375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99627a-8570-4cc5-a57a-e9898f0733c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.506767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 63/138] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:40 2022] PUT /identity/v3/projects/4ff18af8479444da8d5e1f6d00df6351/users/4e79330e00e34ea08d7152d870440c86/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:14:41.520498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ce99627a-8570-4cc5-a57a-e9898f0733c7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:41.522533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ce99627a-8570-4cc5-a57a-e9898f0733c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:41.674309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 76/139] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:41.756358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bef9d-5f70-463f-9627-1097047163af None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.756695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bef9d-5f70-463f-9627-1097047163af None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.756893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bef9d-5f70-463f-9627-1097047163af None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.758321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 64/140] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:41.764539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.764840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.765088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.791408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:41.792684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:41.884944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:41.891391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 77/141] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:42.204318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bef9d-5f70-463f-9627-1097047163af None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:42.332801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:42.369095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bef9d-5f70-463f-9627-1097047163af None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:42.420262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:42.421085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:42.421482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:42.421967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:42.429258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:42.450781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:42.462077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:42.462949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:42.463693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:42.465075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 65/142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:42.476528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:42.752617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0828e9a-4ba6-4028-8e05-6d03da4e6a23 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:42.795264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb901b3-41cb-4756-ab1a-91c4cf7006a8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:42.843146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:42.843894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:42.845312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:42.845668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:42.847318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:42.849099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:42.849775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:42.850066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:42.850395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:42.851279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:42.851590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:42.853234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135231a-9420-4866-accf-f03ec05a15f0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:42.856545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:42.858575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 78/143] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:42.860154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:42.865920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:42.870758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 66/144] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:42.872164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ebdbc-f292-4b2a-bff4-688fa8bde3f2 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:42.876501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ebdbc-f292-4b2a-bff4-688fa8bde3f2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:42.876501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ebdbc-f292-4b2a-bff4-688fa8bde3f2 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:42.876501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 67/145] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:42.880423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ebdbc-f292-4b2a-bff4-688fa8bde3f2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:42.992840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:43.116394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bf5b4-936b-416c-82f9-fffe20ff87f1 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:43.157968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ebdbc-f292-4b2a-bff4-688fa8bde3f2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:43.204675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:43.206276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:43.207130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:43.207742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:43.209043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:43.211559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:43.271613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604d967-122d-464d-ad64-1221d5fc39d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:43.273670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604d967-122d-464d-ad64-1221d5fc39d7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:43.274266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604d967-122d-464d-ad64-1221d5fc39d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:43.274767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604d967-122d-464d-ad64-1221d5fc39d7 None admin] PATH_INFO: `/v3/users/a0909593eafe40ac85627b139c0cfc8d` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:43.276982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604d967-122d-464d-ad64-1221d5fc39d7 None admin] RBAC: Authorizing `identity:get_user(user_id=a0909593eafe40ac85627b139c0cfc8d)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:43.280356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604d967-122d-464d-ad64-1221d5fc39d7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:43.285404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 68/146] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:42 2022] GET /identity/v3/users/a0909593eafe40ac85627b139c0cfc8d => generated 289 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:43.287496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:43.288296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:43.289042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:43.316730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:43.318681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:43.397501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 79/147] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:14:42 2022] POST /identity/v3/users => generated 283 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:14:43.399864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:43.440830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:43.453469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 69/148] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:43.672831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443bd65-43c8-43b3-b57b-6936f6f3df64 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:43.709799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:43.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:43.733392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:43.733719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:43.734029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:43.741349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:43.741349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:43.741349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 80/149] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:43 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:43.751749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:43.946153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec9844-fea7-43dd-9722-897f9b78003d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:43.996613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:43.997338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:43.997615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:43.997884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/a0909593eafe40ac85627b139c0cfc8d/roles/de69739cb03e44e4bad4acc69a2f8e72` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.005951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7fe77a62546939bec3adf10e3790f, user_id=a0909593eafe40ac85627b139c0cfc8d, role_id=de69739cb03e44e4bad4acc69a2f8e72)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:44.007531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:44.035863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:44.196609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 70/150] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:43 2022] PUT /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f/users/a0909593eafe40ac85627b139c0cfc8d/roles/de69739cb03e44e4bad4acc69a2f8e72 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:14:44.241829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f4be7-ba09-45e0-8aec-c06eceedc253 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:44.308381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:44.309097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.309523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.309797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.327455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:44.335025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:44.336659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:44.339378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:44.344764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 81/151] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:44.345011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:44.469071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:44.526240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcef07-ad86-421a-badf-3dc912caa4bb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:44.576889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:44.577552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.577834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.578105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.581792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:44.581792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:44.581792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16efeac-4d6d-4ead-9541-ec3c8fa93ef0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:44.591643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 71/152] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:44.598781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:44.704902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:44.777177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f8e4e-70f6-47f1-bf45-aca84ea85142 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:44.833937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d7510-7690-45ce-b21e-42b7ec1836d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:44.834568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d7510-7690-45ce-b21e-42b7ec1836d6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.834888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d7510-7690-45ce-b21e-42b7ec1836d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.835186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d7510-7690-45ce-b21e-42b7ec1836d6 None admin] PATH_INFO: `/v3/users/3b2336264fcc4528b59320d71b7da06f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.835418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37ac03-e040-444a-89e1-4dd7f2c9cafd None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.835684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37ac03-e040-444a-89e1-4dd7f2c9cafd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.835960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37ac03-e040-444a-89e1-4dd7f2c9cafd None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.836664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d7510-7690-45ce-b21e-42b7ec1836d6 None admin] RBAC: Authorizing `identity:get_user(user_id=3b2336264fcc4528b59320d71b7da06f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:44.837636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 72/153] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:44.838402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d7510-7690-45ce-b21e-42b7ec1836d6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:44.840505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 82/154] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:44 2022] GET /identity/v3/users/3b2336264fcc4528b59320d71b7da06f => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:44.846735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37ac03-e040-444a-89e1-4dd7f2c9cafd None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:44.851241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821555-8e97-496a-b174-224934c2329c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.851622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821555-8e97-496a-b174-224934c2329c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.851926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821555-8e97-496a-b174-224934c2329c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.886832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-df821555-8e97-496a-b174-224934c2329c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:44.888828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-df821555-8e97-496a-b174-224934c2329c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:45.027713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 83/155] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:14:45.038882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37ac03-e040-444a-89e1-4dd7f2c9cafd None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:45.075582 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:45.100745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:45.101529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:45.101892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:45.102257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:45.198131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:45.199624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:45.355236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 73/156] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:45.365548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821555-8e97-496a-b174-224934c2329c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:45.447664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:45.521394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821555-8e97-496a-b174-224934c2329c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:45.596389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07119847-29b4-4c3c-bf4b-513ed45f0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:45.597397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07119847-29b4-4c3c-bf4b-513ed45f0f77 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:45.597897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07119847-29b4-4c3c-bf4b-513ed45f0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:45.598359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07119847-29b4-4c3c-bf4b-513ed45f0f77 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:45.600630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07119847-29b4-4c3c-bf4b-513ed45f0f77 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:45.603273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07119847-29b4-4c3c-bf4b-513ed45f0f77 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:45.606544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 84/157] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:45 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:45.613487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:45.648896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:45.767746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ab790-a67b-41a4-9bf4-bf8a419fdeb7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:45.768261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ab790-a67b-41a4-9bf4-bf8a419fdeb7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:45.768697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ab790-a67b-41a4-9bf4-bf8a419fdeb7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:45.771102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 85/158] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:45.788064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b2ccc-b47a-4e82-bac3-28b5607ae391 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:45.814573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:45.815006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:45.815435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:45.840386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:45.841046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:45.841324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:45.841618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:45.842350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=3b2336264fcc4528b59320d71b7da06f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:45.842651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:45.844517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:45.851987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 74/159] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:45 2022] GET /identity/v3/role_assignments?user.id=3b2336264fcc4528b59320d71b7da06f&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:45.869980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:45.871718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:46.020726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 86/160] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:46.124084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:46.292865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:46.307616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39272983-ff9f-4b5d-af9c-9206d9c83850 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:46.358718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde797b-0c75-49d0-be05-cf8886319904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:46.360763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde797b-0c75-49d0-be05-cf8886319904 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:46.362047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde797b-0c75-49d0-be05-cf8886319904 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:46.362508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde797b-0c75-49d0-be05-cf8886319904 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:46.389572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde797b-0c75-49d0-be05-cf8886319904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-7dde797b-0c75-49d0-be05-cf8886319904 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:46.398994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:46.400720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:46.403814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 75/161] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:46.411511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:46.503866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:46.570990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9494b45-107e-460e-9ee7-63c2995d40dd None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:46.618764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:46.619379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:46.619657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:46.619924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:46.620632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:46.620945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:46.622411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:46.636219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 87/162] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:46.643311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4dfdb-cf64-4213-ba12-346653867c6c None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:46.643674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4dfdb-cf64-4213-ba12-346653867c6c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:46.644110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4dfdb-cf64-4213-ba12-346653867c6c None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:46.645880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:46.650737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 76/163] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:46.659533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:46.659963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:46.660433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:46.706014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:46.708045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:46.799229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cebba-ff43-4bcd-83e6-4ee11be0bbf2 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:46.811803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:46.857100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:46.857782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:46.858065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:46.858335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:46.859628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:46.861218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:46.863141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 88/164] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:46 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:46.871988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:46.933710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 77/165] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:46.950472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:47.146077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef3c28-6a7f-48a1-ad6b-ff50f7a71b55 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:47.216621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:47.255482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:47.256487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:47.257036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:47.257464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:47.321150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262961b1-e5a0-45de-9fcb-025ba04926da None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:47.401286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:47.402274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:47.402408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:47.402774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:47.406723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:47.426334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:47.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:47.439179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:47.442689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 78/166] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:14:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:47.457356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:47.461047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:47.463816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:47.631871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342eb8ba-5b47-463c-b359-a155652eb6fe None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:47.681923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:47.697157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394466-8ab8-469f-8183-470831176359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:47.700634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394466-8ab8-469f-8183-470831176359 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:47.700634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394466-8ab8-469f-8183-470831176359 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:47.700634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394466-8ab8-469f-8183-470831176359 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:47.700634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8394466-8ab8-469f-8183-470831176359 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:47.700634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8394466-8ab8-469f-8183-470831176359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:47.701624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8394466-8ab8-469f-8183-470831176359 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:47.712907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 79/167] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:14:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:47.720494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394466-8ab8-469f-8183-470831176359 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:47.793504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 89/168] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 922 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:47.795190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:47.890538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394466-8ab8-469f-8183-470831176359 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:47.970018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:47.971257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:47.971713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:47.972153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:47.975022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:47.995735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:48.001052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:48.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 80/169] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:14:48.004475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:48.005962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2455dae-b0d8-4fee-a40b-136f9199156a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:48.117033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:48.117639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:48.117989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:48.118257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:48.119523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:48.121163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:48.128353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 90/170] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:47 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:48.128353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:48.199415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:48.331893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ac370-b54d-4601-a11f-5900fa81f42b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:48.423356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf755b5-8eb5-4ca1-b655-0041e8ceb3f3 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:48.425045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:48.425727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:48.426306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:48.426306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:48.429050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:48.430545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:48.432552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 81/171] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:48 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:48.433370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:48.574425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:48.575775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:48.576275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:48.576836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:48.577824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:48.583886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:48.585847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:48.614766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 91/172] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:48.616033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:48.643897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:48.646563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9b456-6cea-41a0-995b-b2b8d7a50f9b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:48.685021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:48.749481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69cd70-93f1-4877-9198-4851ab6076e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:48.752423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69cd70-93f1-4877-9198-4851ab6076e2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:48.752423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69cd70-93f1-4877-9198-4851ab6076e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:48.752423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69cd70-93f1-4877-9198-4851ab6076e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:48.989682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69cd70-93f1-4877-9198-4851ab6076e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:48.993348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69cd70-93f1-4877-9198-4851ab6076e2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:49.028048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bcc74-2956-4e79-8054-3f440d6f7c54 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:49.137452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:49.138434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.138903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.139371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.141183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=4e79330e00e34ea08d7152d870440c86) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:49.141705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:49.144474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:49.160664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 92/173] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:48 2022] GET /identity/v3/role_assignments?user.id=4e79330e00e34ea08d7152d870440c86&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 608 bytes in 541 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:49.162202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:49.247527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 82/174] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 814 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:49.251830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2d574-408d-47b1-b8c4-8366fb1376ad None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.256763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2d574-408d-47b1-b8c4-8366fb1376ad None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.260351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2d574-408d-47b1-b8c4-8366fb1376ad None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.260351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 83/175] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:49.260351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d0a36-5c75-4950-a248-f98270c70ed9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.260351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d0a36-5c75-4950-a248-f98270c70ed9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.260351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d0a36-5c75-4950-a248-f98270c70ed9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.261880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 84/176] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:49.262752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d0a36-5c75-4950-a248-f98270c70ed9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:49.339624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:49.401320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:49.471505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfab861-5a6e-4b97-908c-c49997c955f9 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:49.517960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd318321-0070-49b1-8faf-4071dd324fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:49.518904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd318321-0070-49b1-8faf-4071dd324fe4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.519333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd318321-0070-49b1-8faf-4071dd324fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.519806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd318321-0070-49b1-8faf-4071dd324fe4 None admin] PATH_INFO: `/v3/users/4e79330e00e34ea08d7152d870440c86` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.521582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd318321-0070-49b1-8faf-4071dd324fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=4e79330e00e34ea08d7152d870440c86)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:49.523331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd318321-0070-49b1-8faf-4071dd324fe4 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:49.526000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 93/177] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:49 2022] GET /identity/v3/users/4e79330e00e34ea08d7152d870440c86 => generated 275 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:49.528019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdbad0-2af2-4729-abd6-279d51adffb5 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.528612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdbad0-2af2-4729-abd6-279d51adffb5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.529108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdbad0-2af2-4729-abd6-279d51adffb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.560042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d0a36-5c75-4950-a248-f98270c70ed9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:49.565703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-34bdbad0-2af2-4729-abd6-279d51adffb5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:49.567866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-34bdbad0-2af2-4729-abd6-279d51adffb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:49.617308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b626-315a-460a-af1b-533ee8e14ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:49.619070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b626-315a-460a-af1b-533ee8e14ae3 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.619759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b626-315a-460a-af1b-533ee8e14ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.622201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b626-315a-460a-af1b-533ee8e14ae3 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.625563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b626-315a-460a-af1b-533ee8e14ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:49.628942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b626-315a-460a-af1b-533ee8e14ae3 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:49.632096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 85/178] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:49 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:49.642170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.642579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.643014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.664963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:49.666913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:49.710264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 94/179] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:49.716381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb17e7-c594-4fa5-ad55-d0d8c87a64d1 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:49.716381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb17e7-c594-4fa5-ad55-d0d8c87a64d1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:49.716381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb17e7-c594-4fa5-ad55-d0d8c87a64d1 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:49.716718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 95/180] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:49.718370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb17e7-c594-4fa5-ad55-d0d8c87a64d1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:49.812890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 86/181] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:49.814660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:49.849426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:49.888659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:49.951449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb17e7-c594-4fa5-ad55-d0d8c87a64d1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:49.998156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716114d-42cd-4a37-b535-2ff708d141c9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:50.012192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:50.012859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.013213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.013676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.060984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:50.062687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.063449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.063754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.065907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] RBAC: Adding query filter params (role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=4e79330e00e34ea08d7152d870440c86) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:50.066806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:50.070346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ebffb-bfd0-4728-8b75-caa315cb9508 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:50.080641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 87/182] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:49 2022] GET /identity/v3/role_assignments?user.id=4e79330e00e34ea08d7152d870440c86&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:50.083290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e06b-2b49-45ce-9ac3-77291648e118 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.083769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e06b-2b49-45ce-9ac3-77291648e118 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.084251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e06b-2b49-45ce-9ac3-77291648e118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.113318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e815e06b-2b49-45ce-9ac3-77291648e118 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:50.114743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e815e06b-2b49-45ce-9ac3-77291648e118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:50.161924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:50.164696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:50.227565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 88/183] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:50.230507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3923a-8148-4f07-bd6b-f8c06b7e5b1f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.230738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3923a-8148-4f07-bd6b-f8c06b7e5b1f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.231151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3923a-8148-4f07-bd6b-f8c06b7e5b1f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.235115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 89/184] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:50.236538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3923a-8148-4f07-bd6b-f8c06b7e5b1f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:50.272734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:50.296519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:50.355786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 96/185] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:50.355786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:50.478134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3923a-8148-4f07-bd6b-f8c06b7e5b1f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:50.531359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9a48-2356-4d5c-8298-b6e33be9fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:50.533003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9a48-2356-4d5c-8298-b6e33be9fdaf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.533301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9a48-2356-4d5c-8298-b6e33be9fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.533550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9a48-2356-4d5c-8298-b6e33be9fdaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.546711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd9a48-2356-4d5c-8298-b6e33be9fdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-9fcd9a48-2356-4d5c-8298-b6e33be9fdaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:50.553248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:50.557752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 90/186] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:50.559206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.561288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.561288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.569076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fa5f9-ee94-46c3-aa2d-c8210b605273 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:50.581162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:50.582347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:50.634804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:50.635623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.636050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.636232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.656035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:50.662793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:50.664520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:50.666651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 97/187] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:50.669816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:50.705402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 91/188] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:50.706418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:50.718918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:50.721892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:50.828995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfc4af-cb79-41b3-855d-ae546d1165b3 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:50.880647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:50.880851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d6b7c-77f6-4450-b37f-2aa0256df868 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:50.881746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.882342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.882945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] PATH_INFO: `/v3/projects/4ff18af8479444da8d5e1f6d00df6351` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.885332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] RBAC: Authorizing `identity:get_project(project_id=4ff18af8479444da8d5e1f6d00df6351)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:50.886758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:50.896056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 98/189] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/projects/4ff18af8479444da8d5e1f6d00df6351 => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:50.896056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:50.919945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:50.920615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.920940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.921276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.922512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:50.922977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:50.924656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:50.935161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 92/190] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:50.936032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.025208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.029653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4c162-ca7c-42a6-b903-cf809312f09f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.030727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.066687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.067320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.067588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.067849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.071891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a76c1-3f7f-496e-846a-79ea72c5cf62 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.081086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:51.089113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:51.096638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 99/191] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:51.097769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.113007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.113620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.113893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.114184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.114865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:51.115165 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.116624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.126541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 93/192] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:51.127513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.217093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921bfb7-2c62-4f7b-b988-27fc883d6a18 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.239504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee59a8-4192-47e6-85b2-1d4a7253f812 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.256872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.257483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.257759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.258024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.258749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=4ff18af8479444da8d5e1f6d00df6351, user.id=4e79330e00e34ea08d7152d870440c86) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:51.259050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.260520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.267759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 100/193] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/role_assignments?user.id=4e79330e00e34ea08d7152d870440c86&scope.project.id=4ff18af8479444da8d5e1f6d00df6351&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:51.268374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.288481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.299882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.302161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.302830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.303211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.303494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] PATH_INFO: `/v3/users/b54ffd43713c42b3bfad2dadf56774bd` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.319419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=b54ffd43713c42b3bfad2dadf56774bd)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.320984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.329351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 94/194] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/users/b54ffd43713c42b3bfad2dadf56774bd => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:51.329351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.415895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415fdda-22b1-4c66-b051-5aa408dabf22 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.457600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7fa86-4e4f-4518-be45-22a996c9d7a2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.468868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.469435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.469725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.469966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.471660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.473070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.474474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 101/195] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:51.475034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.508835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.509426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.509705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.509978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.510649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:51.510954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.512390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.522264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 95/196] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:51.523360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.574752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.611020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.624742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0463c-b3e8-459b-a73c-a57b9abae183 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.632634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd4d9-ec81-420b-bfae-6b5341a1efa9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.662403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.663041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.663344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.663618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] PATH_INFO: `/v3/users/a0909593eafe40ac85627b139c0cfc8d` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.664815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] RBAC: Authorizing `identity:get_user(user_id=a0909593eafe40ac85627b139c0cfc8d)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.666301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.667945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 102/197] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/users/a0909593eafe40ac85627b139c0cfc8d => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:51.668828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.670568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.671253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.671537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.671822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.749522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.751649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.787213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c771f-ac88-4d77-9c27-b874cc91b094 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.802155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.826482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.827102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.827378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.827648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.828500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.829984 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.861966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 96/198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:51.862847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.966110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 103/199] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:14:51 2022] POST /identity/v3/users => generated 297 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:51.966110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:51.986554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00da0bb-6a3f-47dd-b85d-65fc76d5d036 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.993080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:52.033627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:52.034315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:52.034602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:52.034874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] PATH_INFO: `/v3/users/b54ffd43713c42b3bfad2dadf56774bd` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:52.036238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] RBAC: Authorizing `identity:get_user(user_id=b54ffd43713c42b3bfad2dadf56774bd)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:52.037883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:52.044716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 97/200] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/users/b54ffd43713c42b3bfad2dadf56774bd => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:52.044716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:52.067938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:52.134221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76594e-ada1-469c-8549-558e3c979c9d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:52.184015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:52.184678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:52.184960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:52.185227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:52.192021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418500c-5b4f-45f1-a006-ee7186175578 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:52.237686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:52.238306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:52.238578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:52.238842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:52.240013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:52.241574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:52.243919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 98/201] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:52 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:52.244848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:52.299980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:52.302771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:52.329418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:52.363372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:52.388394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97ba5b-da78-449a-8ba4-678b69c4415d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:52.482404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:52.483029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:52.483374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:52.483653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:52.490792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 104/202] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:52.491753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:52.580829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:52.583206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:52.613952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaecb4-5c31-4a4d-865b-da14dabda76d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:52.630124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:52.653476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:52.654099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:52.654385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:52.654653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:52.655487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=b54ffd43713c42b3bfad2dadf56774bd) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:52.655799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:52.658082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:52.664208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 105/203] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:52 2022] GET /identity/v3/role_assignments?user.id=b54ffd43713c42b3bfad2dadf56774bd&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:52.669996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:52.727637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 99/204] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:52.729255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:52.741538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:52.847196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d6d1-399a-4da4-aa18-74ce1a979d94 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:52.940026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:52.941227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:52.941684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:52.942181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:52.944214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:52.947180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:52.950580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 106/205] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:52 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:52.959235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:52.962725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a4e1f-6992-4c75-b899-f824edbf6396 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:53.027756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:53.028543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:53.028954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:53.029328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:53.032983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:53.034630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:53.036787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 100/206] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:52 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:53.045348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:53.114626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:53.120009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:53.197661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66a4a-9fb0-49b7-bc58-0887603ae47c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:53.242917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:53.243583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:53.243911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:53.244189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:53.244954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] RBAC: Adding query filter params (role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=b54ffd43713c42b3bfad2dadf56774bd) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:53.245413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:53.247022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd08a21-ea53-4398-9a28-2aa8b2d8727a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:53.255363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 101/207] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:53 2022] GET /identity/v3/role_assignments?user.id=b54ffd43713c42b3bfad2dadf56774bd&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:53.292761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142042c-2e7e-4b1e-b63e-befb8cbe1067 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:53.356125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:53.356764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:53.357039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:53.357307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:53.358033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] RBAC: Adding query filter params (role.id=de69739cb03e44e4bad4acc69a2f8e72, scope.project.id=98d7fe77a62546939bec3adf10e3790f, user.id=a0909593eafe40ac85627b139c0cfc8d) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:53.358335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:53.360737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b6168-6cee-4d0c-8e11-b90198ee4315 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:53.371385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 107/208] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:52 2022] GET /identity/v3/role_assignments?user.id=a0909593eafe40ac85627b139c0cfc8d&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=de69739cb03e44e4bad4acc69a2f8e72 => generated 608 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:54.517891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d9045-c667-433b-9c5d-63a39ff83e03 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:54.518404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d9045-c667-433b-9c5d-63a39ff83e03 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:54.518917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d9045-c667-433b-9c5d-63a39ff83e03 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:54.522061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 102/209] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:54.528925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:54.529398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:54.529734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:54.560205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:54.562502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:54.603455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e84fa-c461-465d-847f-b5282d471035 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:54.603883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e84fa-c461-465d-847f-b5282d471035 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:54.604031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e84fa-c461-465d-847f-b5282d471035 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:54.605475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 103/210] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:54.612660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:54.612929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:54.613190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:54.635893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:54.637142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:54.652635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 108/211] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:54.726480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 104/212] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:54.949377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:54.996409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:55.064157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a3412-278b-4c4a-8636-847d3c1684fd None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:55.111986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:55.113652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.114483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.115300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.132842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:55.139964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:55.140968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:14:55.141982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:55.144473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 109/213] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:55.152442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:55.181725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:55.210777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:55.292621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90b3d-6d67-41f1-bd24-e8db10e2dd2a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:55.341711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:55.345486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:55.346122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.346396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.346665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.347343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:55.347459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf63c16-3116-4862-b6ae-5d2dd5fcedb0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:55.347643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:55.349070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20deb0a-a077-45f6-a004-2ba82bf3938c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:55.358920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 105/214] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:55.361335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb32e14-9765-4ffd-96c1-2ca2c0e1a4ca None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.361335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb32e14-9765-4ffd-96c1-2ca2c0e1a4ca None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.361335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb32e14-9765-4ffd-96c1-2ca2c0e1a4ca None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.365289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 106/215] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:55.368793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.369456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.373940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.390724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:55.392056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:55.410738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:55.411918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.412259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.412705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.414248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:55.416275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:55.419038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 110/216] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:55.420270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:55.502233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 107/217] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:55.502233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:55.565002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:55.606308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0a9d1-9630-4c48-8c5e-ba77669e175a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:55.663018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079cacc-02df-40e0-96d6-26c4072a3599 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:55.664661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad347f8-2293-4942-a4d6-3eb001e4624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:55.665579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad347f8-2293-4942-a4d6-3eb001e4624e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.665971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad347f8-2293-4942-a4d6-3eb001e4624e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.666282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad347f8-2293-4942-a4d6-3eb001e4624e None admin] PATH_INFO: `/v3/users/009aeba64b934d06b091c4cf64b27615` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.681770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad347f8-2293-4942-a4d6-3eb001e4624e None admin] RBAC: Authorizing `identity:get_user(user_id=009aeba64b934d06b091c4cf64b27615)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:55.683245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad347f8-2293-4942-a4d6-3eb001e4624e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:55.685149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 111/218] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:55 2022] GET /identity/v3/users/009aeba64b934d06b091c4cf64b27615 => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:55.686325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355bf42-2629-4363-ab0c-f5052a7ce013 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.686600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355bf42-2629-4363-ab0c-f5052a7ce013 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.686941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355bf42-2629-4363-ab0c-f5052a7ce013 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.688598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 112/219] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:55.694264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.694540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.694812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.717859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:55.720197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:55.731377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:55.732732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:55.733412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:55.734166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:55.735637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:55.738689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:55.748221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:55.825151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 113/220] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:55.826596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:55.908363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 108/221] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:14:55 2022] POST /identity/v3/users => generated 297 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:14:55.908363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:55.994549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d91402-80bb-4cad-9a2c-9dba145e95bc None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.044483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.045194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.045579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.046206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.053010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.066686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c091f-0a9a-46a5-a92e-7b29f70b5e7d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.086676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.133016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.133871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.134277 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.134672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.154879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:56.159134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:56.160625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:56.161473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:14:56.162614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:56.163262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:56.163330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 109/222] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:56.172658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.314302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 114/223] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:14:56.316470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.338873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4cd62-c727-4666-8c16-a54e3ddeb07d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.401439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.402048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.402798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.403073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.403758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:56.404104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:56.405773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:56.408924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.424039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 110/224] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:14:56.424039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.447572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.496208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24ad58-db19-48ce-91ec-e49c43325e01 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.565372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.566267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.566925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.567572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.587191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:56.597215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:56.598991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:14:56.600881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:14:56.602731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341249a-19a5-47ec-8fcc-271eabef7952 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.605378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 115/225] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:56.606937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.654687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.655738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.656699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.657142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.658567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:56.660744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:56.663482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 111/226] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:56.664465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.760877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef618f-45c8-4f0b-a973-4af995f22a52 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.777432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.797080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.801201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec287-9f46-4780-b51e-db048a4201c8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.823146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.823698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.824028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.824243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] PATH_INFO: `/v3/users/b54ffd43713c42b3bfad2dadf56774bd` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.825430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] RBAC: Authorizing `identity:get_user(user_id=b54ffd43713c42b3bfad2dadf56774bd)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:56.827126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:56.831555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 116/227] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/users/b54ffd43713c42b3bfad2dadf56774bd => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:56.833167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.879148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.880294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.880864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.881320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.882550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:56.882674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:56.885657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:56.909106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 112/228] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:56.910401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.975870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b52b4-b32a-4b7c-85d0-77a43172bbb5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:57.017345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:57.018338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:57.018810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:57.019275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:57.020196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=009aeba64b934d06b091c4cf64b27615) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:57.020757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:57.022390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:57.028416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 117/229] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/role_assignments?user.id=009aeba64b934d06b091c4cf64b27615&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:57.029457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:57.084663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:57.142540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f5e19-08e7-42c7-a7fe-fa627ded886f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:57.171830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50213a7-7bf9-4353-9774-79063118ccd6 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:57.185040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:57.186541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:57.187309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:57.188051 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] PATH_INFO: `/v3/users/b54ffd43713c42b3bfad2dadf56774bd` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:57.190669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] RBAC: Authorizing `identity:get_user(user_id=b54ffd43713c42b3bfad2dadf56774bd)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:57.193969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:57.197299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 118/230] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:57 2022] GET /identity/v3/users/b54ffd43713c42b3bfad2dadf56774bd => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:57.221409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:57.252973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:57.254070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:57.254601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:57.255320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:57.257364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:57.264351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:57.268544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 113/231] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:57.280495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:57.303933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:57.451316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c518f-4dfe-4f38-a599-a6c9de023d81 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:57.472219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:57.479204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f7736-5440-4b04-8516-a9a1ecf28ea8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:57.497022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:57.497961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:57.498457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:57.498919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:57.500316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:57.505231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:57.508537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 119/232] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:14:57 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:57.519556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:57.572713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ac4a7-8b96-4a0a-9856-203eee6c822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:57.573607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ac4a7-8b96-4a0a-9856-203eee6c822d None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:57.574013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ac4a7-8b96-4a0a-9856-203eee6c822d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:57.574383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ac4a7-8b96-4a0a-9856-203eee6c822d None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/b54ffd43713c42b3bfad2dadf56774bd/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:57.580502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ac4a7-8b96-4a0a-9856-203eee6c822d None admin] RBAC: Authorizing `identity:create_grant(project_id=3a9e583e2f324455bb4f1fd9b75f60ed, user_id=b54ffd43713c42b3bfad2dadf56774bd, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:57.582539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ac4a7-8b96-4a0a-9856-203eee6c822d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:57.690810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27588e6-55ec-464d-b890-03de2527a362 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:57.696353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 114/233] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:57 2022] PUT /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/b54ffd43713c42b3bfad2dadf56774bd/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:14:57.747444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdd652-7b20-4ebb-8571-c56b558e80dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:57.748588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdd652-7b20-4ebb-8571-c56b558e80dd None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:57.749075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdd652-7b20-4ebb-8571-c56b558e80dd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:57.750088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdd652-7b20-4ebb-8571-c56b558e80dd None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/b54ffd43713c42b3bfad2dadf56774bd/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:57.758667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdd652-7b20-4ebb-8571-c56b558e80dd None admin] RBAC: Authorizing `identity:create_grant(project_id=3a9e583e2f324455bb4f1fd9b75f60ed, user_id=b54ffd43713c42b3bfad2dadf56774bd, role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:57.760516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdd652-7b20-4ebb-8571-c56b558e80dd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:57.845451 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 120/234] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:14:57 2022] PUT /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/b54ffd43713c42b3bfad2dadf56774bd/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:14:58.691498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525de798-43e6-42c6-9007-d3f7cbbc8c8d None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:58.691960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525de798-43e6-42c6-9007-d3f7cbbc8c8d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:58.692097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525de798-43e6-42c6-9007-d3f7cbbc8c8d None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:58.693892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 115/235] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:58.702318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b368cb-0d6c-448b-9bea-894691215665 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:58.703559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b368cb-0d6c-448b-9bea-894691215665 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:58.703559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b368cb-0d6c-448b-9bea-894691215665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:58.727717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-03b368cb-0d6c-448b-9bea-894691215665 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:58.729699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-03b368cb-0d6c-448b-9bea-894691215665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:58.813655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:58.862979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 121/236] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:59.101556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25780f17-3125-4f12-8494-18d8276a0b7d None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:59.102301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25780f17-3125-4f12-8494-18d8276a0b7d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:59.102301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25780f17-3125-4f12-8494-18d8276a0b7d None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.103538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 116/237] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:59.110257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:59.110751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:59.110997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.137407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:59.138631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:59.236595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 122/238] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:14:59.372529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25780f17-3125-4f12-8494-18d8276a0b7d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:59.561805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:59.625765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25780f17-3125-4f12-8494-18d8276a0b7d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:59.653483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:59.708312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:59.708999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:59.709281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:59.709554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.736354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:14:59.742604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:14:59.744268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:14:59.745533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:14:59.759602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 117/239] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:14:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:14:59.763965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:59.846779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64e511-ec0d-45d7-b915-4c49f9996b4f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:59.849371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:59.900116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:59.900719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:59.900999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:59.901272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.902462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:59.903957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:59.905681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 123/240] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:14:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:59.911741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55760573-fa41-4fb3-9df2-3507e612c26e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:59.917390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:59.952938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:59.954256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:59.954744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:59.955266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.956496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:14:59.957019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:59.958755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:59.971907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 118/241] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:14:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:14:59.976404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:59.987420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:00.082268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62a416-7ecd-40d5-a348-fbb5485768f7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:00.135141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97648ad-f2e8-4035-9713-3b50366b0f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:00.136184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97648ad-f2e8-4035-9713-3b50366b0f6d None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.136602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97648ad-f2e8-4035-9713-3b50366b0f6d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.136915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97648ad-f2e8-4035-9713-3b50366b0f6d None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.137599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97648ad-f2e8-4035-9713-3b50366b0f6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:00.139166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97648ad-f2e8-4035-9713-3b50366b0f6d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:00.199223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f618-f3bb-4072-8cbd-1958cd0b50e8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:00.232891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 124/242] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:14:59 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:00.237417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1445e-7085-490b-a777-daebf8ad4403 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.237918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1445e-7085-490b-a777-daebf8ad4403 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.238259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1445e-7085-490b-a777-daebf8ad4403 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.250309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 125/243] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:00 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:00.252655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf612d1-4669-4df1-8a1f-57e1b621a206 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.252915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf612d1-4669-4df1-8a1f-57e1b621a206 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.253678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf612d1-4669-4df1-8a1f-57e1b621a206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.265773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:00.266715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.267126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.267525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] PATH_INFO: `/v3/users/ea93e4491120424983ce7f16686ec657` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.305340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-adf612d1-4669-4df1-8a1f-57e1b621a206 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:00.307420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-adf612d1-4669-4df1-8a1f-57e1b621a206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:00.309614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] RBAC: Authorizing `identity:get_user(user_id=ea93e4491120424983ce7f16686ec657)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:00.311838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:00.315644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 119/244] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:14:59 2022] GET /identity/v3/users/ea93e4491120424983ce7f16686ec657 => generated 297 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:00.331607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:00.351713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:00.426168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 126/245] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:00.688147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcdf1e-87a2-48b8-8991-e3132f778a04 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.688700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcdf1e-87a2-48b8-8991-e3132f778a04 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.688815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcdf1e-87a2-48b8-8991-e3132f778a04 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.691368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 127/246] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:00.698537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.699267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.699381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.791911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:00.793260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:00.795798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:00.886095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730db4e-a1f8-4537-bd24-c2854e56689d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:00.959984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539dd68-7118-4e25-9e1e-a16e1c19f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:00.961195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539dd68-7118-4e25-9e1e-a16e1c19f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:00.961676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539dd68-7118-4e25-9e1e-a16e1c19f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:00.962157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539dd68-7118-4e25-9e1e-a16e1c19f0b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:00.965754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 128/247] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:00.966575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:01.089626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539dd68-7118-4e25-9e1e-a16e1c19f0b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:01.096297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539dd68-7118-4e25-9e1e-a16e1c19f0b3 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:01.152881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:01.177883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:01.201228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac481b-378d-4769-9259-247d09d12a59 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:01.225254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 120/248] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 895 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:01.227278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267c2dc-f5c3-4a2b-a9ce-7bb1f1155244 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.227789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267c2dc-f5c3-4a2b-a9ce-7bb1f1155244 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.228275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267c2dc-f5c3-4a2b-a9ce-7bb1f1155244 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.230179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 121/249] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:01.236523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267c2dc-f5c3-4a2b-a9ce-7bb1f1155244 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:01.261736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042f189-3368-4d73-bb81-a5475bd10953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:01.264569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f189-3368-4d73-bb81-a5475bd10953 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.265209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f189-3368-4d73-bb81-a5475bd10953 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.265936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042f189-3368-4d73-bb81-a5475bd10953 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.293000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042f189-3368-4d73-bb81-a5475bd10953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-a042f189-3368-4d73-bb81-a5475bd10953 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:01.300223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:01.301222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:01.302133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:01.320149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 129/250] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:01.321493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.321780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.322055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.347372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:01.348948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:01.408244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267c2dc-f5c3-4a2b-a9ce-7bb1f1155244 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:01.451591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 130/251] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:01.456413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:01.488752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:01.489669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.490137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.490635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.492106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:01.493842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:01.496368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 122/252] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:01 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:01.498149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:01.541921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:01.682977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314dbf4-fbbc-4a10-a137-bde0f0815837 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:01.686618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:01.741629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b37e2-7f98-4368-9afd-e98ce6940d8c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:01.766104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:01.766752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.767021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.767289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.795747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:01.808938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:01.810748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 123/253] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:01.812267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:01.824596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:01.825498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.825926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.826365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.827380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:01.828525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:01.830506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:01.843884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 131/254] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:01.845402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:01.943030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:01.971445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3abf0-abee-4333-b550-be8cc2514282 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.026074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.027234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.027498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.027868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.028762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] RBAC: Adding query filter params (role.id=de69739cb03e44e4bad4acc69a2f8e72, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=ea93e4491120424983ce7f16686ec657) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:02.029170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.031349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.031862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d6655-ddee-4145-98d6-a07682c79f26 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.048358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 124/255] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:01 2022] GET /identity/v3/role_assignments?user.id=ea93e4491120424983ce7f16686ec657&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=de69739cb03e44e4bad4acc69a2f8e72 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:02.048358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.120104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.120802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.121091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.121350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.149643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:02.154436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:02.160759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:02.163572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 132/256] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:02.164955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.334153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5a6ef-30f8-438c-8fc0-08df2b9df9f4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.401567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8293b3-142a-412c-a402-e38532841e6d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.441138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.441865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.442156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.442440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.443262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:02.443830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.445375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.465621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 125/257] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:02.467492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.473074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.475921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.476457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.476949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] PATH_INFO: `/v3/users/009aeba64b934d06b091c4cf64b27615` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.478754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] RBAC: Authorizing `identity:get_user(user_id=009aeba64b934d06b091c4cf64b27615)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.480488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.482825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 133/258] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/users/009aeba64b934d06b091c4cf64b27615 => generated 297 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:02.484410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.493934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:02.639780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89f8d8-c7be-4439-ad8a-1e3b8811c2a0 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.685963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:02.688146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.689443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.690117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.690570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.691471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:02.691783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.693966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.705779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 126/259] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:02.706674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.745417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea11ef-9681-4307-9090-daec1087607f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.808626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:02.834310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.834983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.835261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.835548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] PATH_INFO: `/v3/users/b54ffd43713c42b3bfad2dadf56774bd` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.836940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=b54ffd43713c42b3bfad2dadf56774bd)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.838443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.846475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 134/260] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/users/b54ffd43713c42b3bfad2dadf56774bd => generated 283 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:02.847639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.854782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98cda3-44d0-4d3d-8358-a1a305bd5405 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.916311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.916919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.917235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.917468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.918955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.920559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.923572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 127/261] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:02.924098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:03.131476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a8d32-d56a-4ee8-ba91-6b644c9f4680 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:03.159520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0ecb1-435c-462a-ba85-917b622b2dc5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:03.175060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6785f70-755b-4169-b092-1cf843b18f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:03.175635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6785f70-755b-4169-b092-1cf843b18f0b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.175922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6785f70-755b-4169-b092-1cf843b18f0b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.176203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6785f70-755b-4169-b092-1cf843b18f0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.181404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:03.254034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:03.254699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.254982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.255391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] PATH_INFO: `/v3/users/b54ffd43713c42b3bfad2dadf56774bd` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.257349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] RBAC: Authorizing `identity:get_user(user_id=b54ffd43713c42b3bfad2dadf56774bd)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:03.258947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:03.261279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 135/262] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/users/b54ffd43713c42b3bfad2dadf56774bd => generated 283 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:03.262498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:03.274084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6785f70-755b-4169-b092-1cf843b18f0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:03.275565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:03.277584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6785f70-755b-4169-b092-1cf843b18f0b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:03.434591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf480d-b000-4960-a8ce-58c21ee3bc05 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:03.458464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 128/263] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:03.458464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c554f-f02e-4263-bacb-a85b30e05a84 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.458464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c554f-f02e-4263-bacb-a85b30e05a84 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.458464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c554f-f02e-4263-bacb-a85b30e05a84 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.458464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 129/264] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:03.458464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c554f-f02e-4263-bacb-a85b30e05a84 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:03.498094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739413e9-8580-4a57-b9c6-d710e823a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:03.498989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739413e9-8580-4a57-b9c6-d710e823a405 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.499953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739413e9-8580-4a57-b9c6-d710e823a405 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.499953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739413e9-8580-4a57-b9c6-d710e823a405 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/009aeba64b934d06b091c4cf64b27615/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.502528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739413e9-8580-4a57-b9c6-d710e823a405 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a9e583e2f324455bb4f1fd9b75f60ed, user_id=009aeba64b934d06b091c4cf64b27615, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:03.504371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739413e9-8580-4a57-b9c6-d710e823a405 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:03.519518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:03.586367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 136/265] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:03 2022] PUT /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/009aeba64b934d06b091c4cf64b27615/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:03.598076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd8ead-6115-45f0-a6c3-d4eaf2a1156f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.598461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd8ead-6115-45f0-a6c3-d4eaf2a1156f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.598778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd8ead-6115-45f0-a6c3-d4eaf2a1156f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.601269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 137/266] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:03.601878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd8ead-6115-45f0-a6c3-d4eaf2a1156f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:03.604032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c554f-f02e-4263-bacb-a85b30e05a84 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:03.678397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe872084-6435-4936-ae60-1936c3fab8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:03.679317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe872084-6435-4936-ae60-1936c3fab8ac None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.680287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe872084-6435-4936-ae60-1936c3fab8ac None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.680777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe872084-6435-4936-ae60-1936c3fab8ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.725609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:03.794350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe872084-6435-4936-ae60-1936c3fab8ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:03.798238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe872084-6435-4936-ae60-1936c3fab8ac None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:03.809833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd8ead-6115-45f0-a6c3-d4eaf2a1156f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:03.845259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:03.879446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0379f30-08a2-4684-9254-a437d0ac49c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:03.880524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0379f30-08a2-4684-9254-a437d0ac49c7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.881023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0379f30-08a2-4684-9254-a437d0ac49c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.882149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0379f30-08a2-4684-9254-a437d0ac49c7 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.884397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0379f30-08a2-4684-9254-a437d0ac49c7 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:03.887090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0379f30-08a2-4684-9254-a437d0ac49c7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:03.890882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 138/267] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:03 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:03.892998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.893474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.893912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:03.923972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:03.926079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:03.966561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 130/268] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:03.969622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:03.970210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:03.970661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:04.001329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:04.003424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:04.047478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 139/269] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:04.048989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:04.162145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:04.185697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 131/270] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:04.187771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:04.267059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3f990-14d7-41ff-805f-8599cc823267 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:04.356246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:04.357307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:04.357796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:04.358271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:04.360662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=b54ffd43713c42b3bfad2dadf56774bd) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:04.361341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:04.364017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:04.375693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 140/271] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:04 2022] GET /identity/v3/role_assignments?user.id=b54ffd43713c42b3bfad2dadf56774bd&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:04.426040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:04.498697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a92ad-76b6-41c8-b7b0-f482987ac1f3 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:04.549294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:04.549882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:04.550161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:04.550429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:04.557338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:04.558901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:04.560737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 132/272] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:04 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:04.570027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:04.667680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:04.827065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bdd40-f3e1-43db-9839-3a2a2fbedd0d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:04.853375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06354871-4c35-417c-911b-dd4d09c5088a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:04.867295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:04.880626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:04.881382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:04.882292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:04.882292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:04.883059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] RBAC: Adding query filter params (role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=b54ffd43713c42b3bfad2dadf56774bd) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:04.883207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:04.885589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:04.893411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 141/273] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:04 2022] GET /identity/v3/role_assignments?user.id=b54ffd43713c42b3bfad2dadf56774bd&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:04.894376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:04.926707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f70cd-6ada-4117-ab5b-37d01ca17ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:04.927887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f70cd-6ada-4117-ab5b-37d01ca17ad5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:04.928433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f70cd-6ada-4117-ab5b-37d01ca17ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:04.928908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f70cd-6ada-4117-ab5b-37d01ca17ad5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:04.930437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f70cd-6ada-4117-ab5b-37d01ca17ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:04.933081 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f70cd-6ada-4117-ab5b-37d01ca17ad5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:04.935381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 133/274] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:04.939025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b11f-9a21-4dcb-9380-03cae049a70f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:04.939704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b11f-9a21-4dcb-9380-03cae049a70f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:04.940166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b11f-9a21-4dcb-9380-03cae049a70f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:04.943994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 134/275] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:04.947024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b11f-9a21-4dcb-9380-03cae049a70f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:04.955540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:05.069963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9aee9-65ed-49c7-b9e1-ba5a8ee9fa63 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:05.127851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839e538-a608-4071-8a5f-06b7443e1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:05.129449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839e538-a608-4071-8a5f-06b7443e1429 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.129449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839e538-a608-4071-8a5f-06b7443e1429 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.129871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839e538-a608-4071-8a5f-06b7443e1429 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.145404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b11f-9a21-4dcb-9380-03cae049a70f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:05.153546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839e538-a608-4071-8a5f-06b7443e1429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e839e538-a608-4071-8a5f-06b7443e1429 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:05.159325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:05.160846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:05.162080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 142/276] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:05.163762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.163762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.163762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.185340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:05.185340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:05.215088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:05.215770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.216050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.216949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.217286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:05.220564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:05.258614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:05.286778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 143/277] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:05.287638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:05.357409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 135/278] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:15:04 2022] POST /identity/v3/users => generated 293 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:05.474387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b9d78-d3d0-4ffd-92e0-a62a0dc4a82a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:05.537839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:05.538648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.539017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.539395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.540405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:05.540935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:05.542490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:05.553314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 144/279] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:05.559250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:05.639599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:05.641594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:05.705145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d039400-f856-486c-89fc-b6f8a3a31f69 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:05.719226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:05.777468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:05.777468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.777468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.777468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] PATH_INFO: `/v3/users/0286e35cd4604d208f8b528e84cadf82` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.794970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] RBAC: Authorizing `identity:get_user(user_id=0286e35cd4604d208f8b528e84cadf82)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:05.797687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:05.799297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 136/280] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:05 2022] GET /identity/v3/users/0286e35cd4604d208f8b528e84cadf82 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:05.805609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:05.819870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e8ddc-37ca-45ea-84cc-c62879886dc0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:05.863939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:05.865237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.865746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.866243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.887564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:05.899342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:05.900294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:05.901394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:05.902476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 145/281] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:05.910236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:05.930263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b56d6-c99e-4cd3-9e1f-04cbd24f7d00 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:05.950681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:05.976840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:05.977594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:05.977874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:05.978214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:05.978952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, user.id=0286e35cd4604d208f8b528e84cadf82, scope.system=all) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:05.979256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:05.980930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:05.988318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 137/282] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 00:15:05 2022] GET /identity/v3/role_assignments?user.id=0286e35cd4604d208f8b528e84cadf82&scope.system=all&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:06.049052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795b7f4-236e-4889-85eb-bb3f30d8ea1f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:06.051219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:06.106127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:06.106760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:06.107041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:06.107309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:06.107978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:06.108274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:06.110414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10c9ec-0393-423a-805f-e964d917b34a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:06.125763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 146/283] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:06.132498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:06.294951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4be012-07ff-4cf8-a85a-72349446b5e4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:06.351932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:06.352536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:06.352810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:06.353075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] PATH_INFO: `/v3/users/ea93e4491120424983ce7f16686ec657` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:06.354366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] RBAC: Authorizing `identity:get_user(user_id=ea93e4491120424983ce7f16686ec657)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:06.355833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:06.358385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 138/284] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:06 2022] GET /identity/v3/users/ea93e4491120424983ce7f16686ec657 => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:06.360831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb523dd8-7605-4d29-86cc-ac2e467af99e None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:06.361564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb523dd8-7605-4d29-86cc-ac2e467af99e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:06.361564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb523dd8-7605-4d29-86cc-ac2e467af99e None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:06.363134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 147/285] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:15:06.365531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:06.368749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:06.369031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:06.369299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:06.396640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:06.398623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:06.475714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:06.539150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 148/286] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:06.585739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1177ba-999b-48f0-a753-020389934f3d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:06.673103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:06.673820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:06.674266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:06.674622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:06.676206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:06.678146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:06.680693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 139/287] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:06 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:06.689931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:06.904473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:06.912518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:06.925621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2480c-6645-4336-8a6f-940cbeb177d1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:07.018135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:07.019342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:07.019857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:07.020458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/ea93e4491120424983ce7f16686ec657/roles/de69739cb03e44e4bad4acc69a2f8e72` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:07.034621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] RBAC: Authorizing `identity:create_grant(project_id=3a9e583e2f324455bb4f1fd9b75f60ed, user_id=ea93e4491120424983ce7f16686ec657, role_id=de69739cb03e44e4bad4acc69a2f8e72)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:07.037322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:07.123305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dab02b-4318-46a3-b489-f4a5700183a7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:07.135842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 149/288] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:06 2022] PUT /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/users/ea93e4491120424983ce7f16686ec657/roles/de69739cb03e44e4bad4acc69a2f8e72 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:07.149017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:07.180574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:07.181318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:07.181599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:07.181864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:07.199479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:07.205503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:07.207129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:07.208677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:07.216373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 140/289] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:07.216680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:07.468055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47566432-9cf9-4238-8817-0f1e0aa3e9be None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:07.511172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:07.511796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:07.512170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:07.512693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:07.513941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:07.514418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:07.517810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:07.531872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 150/290] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:07.537622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:07.739803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8262565-d191-490c-a1a8-dfe5099c4c6c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:07.825191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:07.826412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:07.827007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:07.828672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] PATH_INFO: `/v3/users/009aeba64b934d06b091c4cf64b27615` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:07.830901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] RBAC: Authorizing `identity:get_user(user_id=009aeba64b934d06b091c4cf64b27615)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:07.833677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:07.837185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 141/291] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:07 2022] GET /identity/v3/users/009aeba64b934d06b091c4cf64b27615 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:07.846221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:07.876859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:07.991541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbf896-6cd1-4af2-91e8-93ec2442953d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:08.036506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:08.037132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.037441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.037672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.170975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:08.173576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:08.208663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:08.357528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 151/292] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:08.365231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:08.455764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:08.536677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c97d0-cc85-49a8-bde4-cd21644c8e35 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:08.604108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5ae7-8588-4302-b1c4-effc7960957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:08.604790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5ae7-8588-4302-b1c4-effc7960957b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.605063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5ae7-8588-4302-b1c4-effc7960957b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.605330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5ae7-8588-4302-b1c4-effc7960957b None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.606533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c5ae7-8588-4302-b1c4-effc7960957b None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:08.608044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c5ae7-8588-4302-b1c4-effc7960957b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:08.610119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 142/293] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:08 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:08.616108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:08.668392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23881-5c83-4575-95be-c467649887a9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.668392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23881-5c83-4575-95be-c467649887a9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.668392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23881-5c83-4575-95be-c467649887a9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.668942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 143/294] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:15:08.688854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88b0f9-4e25-4f55-8cf3-f3a6120f326b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.689339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88b0f9-4e25-4f55-8cf3-f3a6120f326b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.689839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88b0f9-4e25-4f55-8cf3-f3a6120f326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.716276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4b88b0f9-4e25-4f55-8cf3-f3a6120f326b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:08.718888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4b88b0f9-4e25-4f55-8cf3-f3a6120f326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:08.823286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3eb883-d26c-403a-b350-914061caf69e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:08.830861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 144/295] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:08.832360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa2017-6d2e-4ea6-bea0-0dd06ad320f9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.833342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa2017-6d2e-4ea6-bea0-0dd06ad320f9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.833342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa2017-6d2e-4ea6-bea0-0dd06ad320f9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.840070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 145/296] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:08.842618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dce013-cbef-43c9-ac93-c3bd6d35d6c6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.843335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dce013-cbef-43c9-ac93-c3bd6d35d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.843774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:08.844030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dce013-cbef-43c9-ac93-c3bd6d35d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.864944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-34dce013-cbef-43c9-ac93-c3bd6d35d6c6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:08.866305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:08.866451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-34dce013-cbef-43c9-ac93-c3bd6d35d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:08.867258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:08.867674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:08.868154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:08.869276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=009aeba64b934d06b091c4cf64b27615) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:08.869828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:08.872506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:08.881599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 152/297] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:08 2022] GET /identity/v3/role_assignments?user.id=009aeba64b934d06b091c4cf64b27615&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:08.948827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:08.998727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 146/298] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:09.229257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:09.336078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7680a4-1196-4ce1-9ea8-629d34ba24dd None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:09.336078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7680a4-1196-4ce1-9ea8-629d34ba24dd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:09.336078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7680a4-1196-4ce1-9ea8-629d34ba24dd None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:09.337373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 147/299] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:15:09.348717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:09.349021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:09.349521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:09.371328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24becbb6-2c92-4dbe-9956-814a71c96406 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:09.387919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:09.390121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:09.422346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:09.423006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:09.423274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:09.423749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:09.426184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:09.427677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:09.429387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 153/300] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:09.436015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:09.512377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 148/301] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:09.512377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:09.527795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:09.597039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d304-94bf-4949-abc2-e875328416b5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:09.646216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:09.647186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:09.647788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:09.648143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] PATH_INFO: `/v3/groups` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:09.649090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:09.651101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:09.664867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a125670-b664-427a-9794-176c58bc0a09 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:09.665495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:09.707745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:09.708998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:09.709385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:09.709796 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:09.722699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:09.727797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:09.728869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:09.737355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 149/302] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:09.739937 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:09.758705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 154/303] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 31 00:15:09 2022] POST /identity/v3/groups => generated 229 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:09.826318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:09.917281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6884d4f-e14a-4fa1-a159-bc776b9af7b4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:09.988777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:09.989595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:09.989879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:09.990154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:09.990835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:09.991306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:09.994418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:10.000500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe20b9-9eba-4ba0-a9e9-0fb887af67cc None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.004207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 150/304] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:10.012418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:10.021704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:10.059459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:10.072712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec83ce2-059f-4eb2-9a17-294fd3a72c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.073730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec83ce2-059f-4eb2-9a17-294fd3a72c51 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.074237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec83ce2-059f-4eb2-9a17-294fd3a72c51 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.074702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec83ce2-059f-4eb2-9a17-294fd3a72c51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.092112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec83ce2-059f-4eb2-9a17-294fd3a72c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-dec83ce2-059f-4eb2-9a17-294fd3a72c51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:10.097882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:10.098956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:10.099970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:10.102294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 155/305] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:10.103726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078eeb2-bd07-4481-9ce0-dbaf95554dde None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.104031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078eeb2-bd07-4481-9ce0-dbaf95554dde None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.104292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078eeb2-bd07-4481-9ce0-dbaf95554dde None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.109707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 156/306] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:10.109707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078eeb2-bd07-4481-9ce0-dbaf95554dde None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:10.180662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36abd0-39d1-4429-8316-9ce7c45c598e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.229258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3da1-803b-4bd4-a6d4-e38753f5b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.229903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3da1-803b-4bd4-a6d4-e38753f5b553 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.230175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3da1-803b-4bd4-a6d4-e38753f5b553 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.230443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3da1-803b-4bd4-a6d4-e38753f5b553 None admin] PATH_INFO: `/v3/users/a2ee434d98c04f0b8d157ce8dfbf7102` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.232300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078eeb2-bd07-4481-9ce0-dbaf95554dde None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.247593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3da1-803b-4bd4-a6d4-e38753f5b553 None admin] RBAC: Authorizing `identity:get_user(user_id=a2ee434d98c04f0b8d157ce8dfbf7102)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:10.249965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3da1-803b-4bd4-a6d4-e38753f5b553 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:10.256700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 151/307] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:10 2022] GET /identity/v3/users/a2ee434d98c04f0b8d157ce8dfbf7102 => generated 293 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:10.256700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.256700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.256700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.276842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:10.278184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:10.286553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.287167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.287914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.288243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.288974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:10.289286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:10.291504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:10.302570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 157/308] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:10.305443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:10.346623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:10.362096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 152/309] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:10.363420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:10.382676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:10.442220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396b8a2-bc49-4b06-8e45-3d86f45168c8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.497771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.499718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.500788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.501460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.502921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb9c3e-99cf-4377-98d4-59d17e634619 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.503344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] RBAC: Adding query filter params (role.id=de69739cb03e44e4bad4acc69a2f8e72, user.id=a2ee434d98c04f0b8d157ce8dfbf7102, scope.system=all) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:10.504114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:10.508254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:10.520839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 158/310] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 00:15:10 2022] GET /identity/v3/role_assignments?user.id=a2ee434d98c04f0b8d157ce8dfbf7102&scope.system=all&role.id=de69739cb03e44e4bad4acc69a2f8e72 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:10.563962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.564983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.565464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.565994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] PATH_INFO: `/v3/users/0286e35cd4604d208f8b528e84cadf82` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.567466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] RBAC: Authorizing `identity:get_user(user_id=0286e35cd4604d208f8b528e84cadf82)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:10.569557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:10.571868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 153/311] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:10 2022] GET /identity/v3/users/0286e35cd4604d208f8b528e84cadf82 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:10.578989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:10.665449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:10.717599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:10.734979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dc160-f969-45eb-95ba-caf8ba529b91 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.788954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.790131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.791030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.791471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] PATH_INFO: `/v3/system/users/0286e35cd4604d208f8b528e84cadf82/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.793651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0286e35cd4604d208f8b528e84cadf82, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:10.795509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:10.836029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:10.870371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b224fb-c98a-4a49-af1e-3d8bcbbbff3a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:10.920879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:10.921916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:10.922046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:10.922390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:10.936955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 159/312] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 31 00:15:10 2022] PUT /identity/v3/system/users/0286e35cd4604d208f8b528e84cadf82/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:10.953446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:10.963749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:10.965483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:10.966977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:10.969996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 154/313] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:10.979269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:11.160045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b7f73-06b3-40a2-b10b-d57940acad6b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:11.204644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:11.205667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:11.206151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:11.206653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:11.207596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:11.208099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:11.209856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:11.220389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 160/314] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:11.232385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:11.357511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834055-e6dc-4736-8b68-6428183aa955 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:11.395995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:11.396663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:11.396943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:11.397214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] PATH_INFO: `/v3/users/ea93e4491120424983ce7f16686ec657` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:11.398398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] RBAC: Authorizing `identity:get_user(user_id=ea93e4491120424983ce7f16686ec657)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:11.400119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:11.401932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 155/315] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:11 2022] GET /identity/v3/users/ea93e4491120424983ce7f16686ec657 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:11.409282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:11.439780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:11.546531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eb4a0-d384-4b3e-bcf8-021fc2f57fb0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:11.589551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:11.590580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:11.591079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:11.591604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:11.680911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:11.682449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:11.707388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:11.799465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 161/316] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:11.805500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:11.817659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:11.933667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19291aea-daa9-4bf1-9df6-75e2e46fc8fc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:11.971417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e5e0-8cf7-4046-bd5c-efed497dedbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:11.972010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e5e0-8cf7-4046-bd5c-efed497dedbd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:11.972339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e5e0-8cf7-4046-bd5c-efed497dedbd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:11.972623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e5e0-8cf7-4046-bd5c-efed497dedbd None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:11.973833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337e5e0-8cf7-4046-bd5c-efed497dedbd None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:11.975399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337e5e0-8cf7-4046-bd5c-efed497dedbd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:11.977496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 156/317] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:11 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:11.984606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:12.113790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255c205-ced8-4097-80bd-0dbc55bb473c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:12.137849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:12.156274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:12.156917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:12.161233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:12.161233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:12.161233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] RBAC: Adding query filter params (role.id=de69739cb03e44e4bad4acc69a2f8e72, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed, user.id=ea93e4491120424983ce7f16686ec657) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:12.161233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:12.161233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b2c-1ad2-49bb-a26e-66c7d8af93ac None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:12.167073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 162/318] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:11 2022] GET /identity/v3/role_assignments?user.id=ea93e4491120424983ce7f16686ec657&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=de69739cb03e44e4bad4acc69a2f8e72 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:12.176364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad4e49-c828-4013-8848-2f79959f5e17 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:12.176679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad4e49-c828-4013-8848-2f79959f5e17 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:12.176914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad4e49-c828-4013-8848-2f79959f5e17 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:12.178648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 157/319] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:12.185489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95a10d-3842-4611-96b0-c8da5595c2f1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:12.185945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95a10d-3842-4611-96b0-c8da5595c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:12.186414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95a10d-3842-4611-96b0-c8da5595c2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:12.220884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6d95a10d-3842-4611-96b0-c8da5595c2f1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:12.222967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6d95a10d-3842-4611-96b0-c8da5595c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:12.333396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 163/320] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:12.802283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad4e49-c828-4013-8848-2f79959f5e17 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:12.841728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d19df9-9e80-436f-9834-1908f93a6a2d None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:12.842269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d19df9-9e80-436f-9834-1908f93a6a2d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:12.842269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d19df9-9e80-436f-9834-1908f93a6a2d None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:12.844643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 164/321] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:15:12.851150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:12.851493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:12.851705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:12.871087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:12.872299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:12.946590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:12.977458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 165/322] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:12.992560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad4e49-c828-4013-8848-2f79959f5e17 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:13.060279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:13.061277 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:13.061799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:13.062387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:13.064440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:13.067240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:13.070843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 158/323] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:13.080381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:13.174174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:13.279251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be475fd-5010-4dc1-8c93-a14033500d4c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:13.341463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ecb37-1269-4285-9fb5-f61b69e65b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:13.342368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ecb37-1269-4285-9fb5-f61b69e65b99 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:13.342894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ecb37-1269-4285-9fb5-f61b69e65b99 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:13.343381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ecb37-1269-4285-9fb5-f61b69e65b99 None admin] PATH_INFO: `/v3/groups` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:13.344691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ecb37-1269-4285-9fb5-f61b69e65b99 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:13.347561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ecb37-1269-4285-9fb5-f61b69e65b99 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:13.415860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:13.492430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 166/324] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 31 00:15:13 2022] POST /identity/v3/groups => generated 226 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:13.494749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543f277-03ea-4fe1-81e1-fda440bbac74 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:13.495224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543f277-03ea-4fe1-81e1-fda440bbac74 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:13.495690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543f277-03ea-4fe1-81e1-fda440bbac74 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:13.497963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 167/325] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:13.506855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:13.507181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:13.507580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:13.536999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:13.539167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:13.564244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43770c-6e4f-49cb-aeed-810222753a93 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:13.610246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:13.610802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:13.611078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:13.611347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:13.625554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:13.633392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:13.634413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:13.635211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 159/326] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:13.641855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:13.646217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:13.659905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 168/327] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:13.763679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078dd7a-ba13-4e5c-bfa0-adde59b09ef8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:13.825958 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:13.826924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:13.827918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:13.828564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:13.829763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] RBAC: Adding query filter params (name=reader) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:13.830169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:13.832978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:13.848845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 160/328] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:13.853827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:13.898035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:13.948457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:13.992134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e3772-3e44-4d69-bc28-280c1e8b5860 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:14.047817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:14.048888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:14.048888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:14.049317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] PATH_INFO: `/v3/users/a2ee434d98c04f0b8d157ce8dfbf7102` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:14.050902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:14.051024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] RBAC: Authorizing `identity:get_user(user_id=a2ee434d98c04f0b8d157ce8dfbf7102)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:14.053094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:14.056241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 169/329] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:13 2022] GET /identity/v3/users/a2ee434d98c04f0b8d157ce8dfbf7102 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:14.062224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:14.116453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3472c-3657-4dbb-a51b-159026068cce None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:14.185084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:14.186184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:14.187446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:14.188033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:14.199335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62699-d180-4adc-86c1-8467bbd1cae6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:14.201453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:14.209244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:14.220204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:14.221928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:14.223472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 161/330] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:14.231544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:14.250701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:14.251385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:14.251663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:14.251936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] PATH_INFO: `/v3/system/users/a2ee434d98c04f0b8d157ce8dfbf7102/roles/de69739cb03e44e4bad4acc69a2f8e72` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:14.253527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a2ee434d98c04f0b8d157ce8dfbf7102, role_id=de69739cb03e44e4bad4acc69a2f8e72)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:14.255541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:14.354425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 170/331] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 31 00:15:14 2022] PUT /identity/v3/system/users/a2ee434d98c04f0b8d157ce8dfbf7102/roles/de69739cb03e44e4bad4acc69a2f8e72 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:14.421164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29111897-c1af-477d-9766-95f49cdefb56 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:14.463302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:14.481008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:14.481955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:14.482406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:14.482816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:14.484175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:14.484589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:14.488360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:14.504854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 162/332] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:14.513728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:14.681402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc05e1-b9e5-4044-b296-03985bf22e37 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:14.737154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf247c1-cee4-40c1-a3e2-8910d5b8ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:14.737731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf247c1-cee4-40c1-a3e2-8910d5b8ea71 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:14.738055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf247c1-cee4-40c1-a3e2-8910d5b8ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:14.738324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf247c1-cee4-40c1-a3e2-8910d5b8ea71 None admin] PATH_INFO: `/v3/users/0286e35cd4604d208f8b528e84cadf82` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:14.739604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf247c1-cee4-40c1-a3e2-8910d5b8ea71 None admin] RBAC: Authorizing `identity:get_user(user_id=0286e35cd4604d208f8b528e84cadf82)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:14.741248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf247c1-cee4-40c1-a3e2-8910d5b8ea71 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:14.742909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 171/333] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:14 2022] GET /identity/v3/users/0286e35cd4604d208f8b528e84cadf82 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:14.753792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:14.998170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae4e58-346e-40e2-90bb-b4034d4022fc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:15.064637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:15.065624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:15.066138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:15.066733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:15.068285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] RBAC: Adding query filter params (role.id=c326d7ede2124b9e9b3714569c80f57e, user.id=0286e35cd4604d208f8b528e84cadf82, scope.system=all) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:15.068844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:15.071674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e587e-0fb0-44e7-9132-b0e374137542 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:15.096395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 163/334] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 00:15:14 2022] GET /identity/v3/role_assignments?user.id=0286e35cd4604d208f8b528e84cadf82&scope.system=all&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 510 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:16.316317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762533c1-8240-4a7d-b3a6-de6eef020d2d None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:16.316875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762533c1-8240-4a7d-b3a6-de6eef020d2d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:16.316875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762533c1-8240-4a7d-b3a6-de6eef020d2d None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:16.318804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 172/335] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:16.324312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0341f-0c08-4434-99dc-14e4f2029a38 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:16.324776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0341f-0c08-4434-99dc-14e4f2029a38 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:16.324957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0341f-0c08-4434-99dc-14e4f2029a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:16.349278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9de0341f-0c08-4434-99dc-14e4f2029a38 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:16.350515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9de0341f-0c08-4434-99dc-14e4f2029a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:16.410625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:16.442333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 164/336] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:17.161969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762533c1-8240-4a7d-b3a6-de6eef020d2d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:17.207075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:17.250058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22617a12-042d-4684-9794-1ad7a9996f03 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.250691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22617a12-042d-4684-9794-1ad7a9996f03 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.251170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22617a12-042d-4684-9794-1ad7a9996f03 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:17.252827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 165/337] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:17.261871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.262618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.263107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:17.298481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:17.300103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:17.307778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762533c1-8240-4a7d-b3a6-de6eef020d2d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:17.392133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:17.397214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.398853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.399155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:17.429427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:17.439344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:17.440296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:17.441115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:17.445309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 173/338] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:17.449092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:17.466484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 166/339] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:17.651283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:17.660077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1352500-2b5a-4497-9065-daac6b7aefe0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:17.703897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:17.704530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.704808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.705083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:17.705835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:17.706165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:17.707763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2251f-eb9b-471b-994f-f6683e190579 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:17.718918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 174/340] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:17.724180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:17.852060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e74a0c-a22b-4efe-b12f-19c2b1b9aa66 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:17.857881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb26e-ae25-48b7-8b46-d329e4bdcd57 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.858972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb26e-ae25-48b7-8b46-d329e4bdcd57 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.859219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb26e-ae25-48b7-8b46-d329e4bdcd57 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:17.860834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 175/341] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:17.866213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:17.874259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.874598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.874844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:17.905514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:17.906914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:17.930763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:17.931394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:17.931670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:17.934074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.061036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 176/342] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:18.062761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.083609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:18.085134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:18.221618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:18.245172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c1ef6-e204-47d4-830c-b3864ff2699c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:18.247883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 167/343] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:18.254584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.266827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:18.317587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9d3bf-2865-42c8-a5cb-b86799b32812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:18.318457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9d3bf-2865-42c8-a5cb-b86799b32812 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.318731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9d3bf-2865-42c8-a5cb-b86799b32812 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.319000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9d3bf-2865-42c8-a5cb-b86799b32812 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.348753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9d3bf-2865-42c8-a5cb-b86799b32812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-04e9d3bf-2865-42c8-a5cb-b86799b32812 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:18.354436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:18.355390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:15:18.356256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:18.358503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 177/344] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:18.370765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8b249-a4a6-484e-85ce-2d6203bd93b1 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.371073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8b249-a4a6-484e-85ce-2d6203bd93b1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.371298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8b249-a4a6-484e-85ce-2d6203bd93b1 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.375397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 178/345] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:18 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:18.379974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8b249-a4a6-484e-85ce-2d6203bd93b1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.504240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b077109-a619-47c8-8853-dbb0528ec71a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:18.557337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a37a63-02c6-452a-b8d4-52ce9a9dfe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:18.558013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a37a63-02c6-452a-b8d4-52ce9a9dfe54 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.558285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a37a63-02c6-452a-b8d4-52ce9a9dfe54 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.558551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a37a63-02c6-452a-b8d4-52ce9a9dfe54 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.559847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a37a63-02c6-452a-b8d4-52ce9a9dfe54 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:18.562384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a37a63-02c6-452a-b8d4-52ce9a9dfe54 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:18.564645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 168/346] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:18.566293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.566560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.566865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.581508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8b249-a4a6-484e-85ce-2d6203bd93b1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:18.593644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:18.594930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:18.629448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:18.630146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.630422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.630736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.631410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:18.631707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:18.633258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:18.644808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 179/347] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:18.645957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.661507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:18.675408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 169/348] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:18.676091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.680572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:18.776585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d58fe-2c1f-41d9-b886-735dc70e6364 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:18.790792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b5435-684d-480c-a45c-6759c424a5ef None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:18.816481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:18.817540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.818164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.818520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.832426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:18.833659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a244cc-750d-4d70-a41a-8aef5c3e08d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:18.834305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a244cc-750d-4d70-a41a-8aef5c3e08d6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.834578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a244cc-750d-4d70-a41a-8aef5c3e08d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.834844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a244cc-750d-4d70-a41a-8aef5c3e08d6 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:18.838758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:18.841154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:18.843722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 180/349] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:18.845516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a244cc-750d-4d70-a41a-8aef5c3e08d6 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:18.845516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a244cc-750d-4d70-a41a-8aef5c3e08d6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:18.845789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.845927 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 170/350] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:18.852369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6e7d7-ea7c-4887-97d0-e4666fe9d932 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:18.852369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6e7d7-ea7c-4887-97d0-e4666fe9d932 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:18.852369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6e7d7-ea7c-4887-97d0-e4666fe9d932 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:18.852369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 171/351] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:18.852369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6e7d7-ea7c-4887-97d0-e4666fe9d932 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:18.928152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:18.930007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:18.970006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c886-6b56-442c-9c67-a1cc8f03d5a1 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:18.971504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6e7d7-ea7c-4887-97d0-e4666fe9d932 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:19.011528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb2893-c9bd-4b0f-96c3-88ac6ba19955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.011864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.012170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb2893-c9bd-4b0f-96c3-88ac6ba19955 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.012480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb2893-c9bd-4b0f-96c3-88ac6ba19955 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.012650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.012793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb2893-c9bd-4b0f-96c3-88ac6ba19955 None admin] PATH_INFO: `/v3/users/a2ee434d98c04f0b8d157ce8dfbf7102` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.012934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.013080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.013788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:19.013959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb2893-c9bd-4b0f-96c3-88ac6ba19955 None admin] RBAC: Authorizing `identity:get_user(user_id=a2ee434d98c04f0b8d157ce8dfbf7102)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:19.014102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:19.015998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb2893-c9bd-4b0f-96c3-88ac6ba19955 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:19.016187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:19.018380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 181/352] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/users/a2ee434d98c04f0b8d157ce8dfbf7102 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:19.019598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.019875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.020171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.026521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 172/353] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:19.027563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.044603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:19.046490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:19.144949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 182/354] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:19.145737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.184761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5356e-50db-4ec6-8306-e557ac0caff3 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:19.230437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:19.236469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:19.237611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.238609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.238609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.238823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.239758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=98d7fe77a62546939bec3adf10e3790f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:19.240108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:19.241614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:19.252437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 173/355] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:19.253651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.293019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d7212-b6fe-4d44-9f2b-15f698cf19cd None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:19.353505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.354156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.355163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.355466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.356248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] RBAC: Adding query filter params (role.id=de69739cb03e44e4bad4acc69a2f8e72, user.id=a2ee434d98c04f0b8d157ce8dfbf7102, scope.system=all) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:19.356593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:19.359275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:19.380295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 183/356] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/role_assignments?user.id=a2ee434d98c04f0b8d157ce8dfbf7102&scope.system=all&role.id=de69739cb03e44e4bad4acc69a2f8e72 => generated 510 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:19.381136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.405320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9059a-92b4-4040-8fcf-caaa9d02add6 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:19.458442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d522c-16d0-43d1-a5a4-9edc8c1f35ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.459131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d522c-16d0-43d1-a5a4-9edc8c1f35ac None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.459409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d522c-16d0-43d1-a5a4-9edc8c1f35ac None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.459678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d522c-16d0-43d1-a5a4-9edc8c1f35ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.478127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d522c-16d0-43d1-a5a4-9edc8c1f35ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e12d522c-16d0-43d1-a5a4-9edc8c1f35ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:19.489033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:19.489866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 174/357] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:19.490679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be467-5ad8-4b5f-b5bc-18b86e93d4da None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.491460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be467-5ad8-4b5f-b5bc-18b86e93d4da None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.491460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be467-5ad8-4b5f-b5bc-18b86e93d4da None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.491460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 175/358] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:19.491460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7be467-5ad8-4b5f-b5bc-18b86e93d4da None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.575808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b250e-0575-4c8a-ac86-02a3a4a17566 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:19.607892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:19.630000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.631162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.632036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.632528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.634633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:19.728173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7be467-5ad8-4b5f-b5bc-18b86e93d4da None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:19.748738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:19.750926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:19.775054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daaa2a-6b42-4ad3-a5a3-b7bc3ff14888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:19.776097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daaa2a-6b42-4ad3-a5a3-b7bc3ff14888 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.776622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daaa2a-6b42-4ad3-a5a3-b7bc3ff14888 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.777128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daaa2a-6b42-4ad3-a5a3-b7bc3ff14888 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.800419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daaa2a-6b42-4ad3-a5a3-b7bc3ff14888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-70daaa2a-6b42-4ad3-a5a3-b7bc3ff14888 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:19.806251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:19.807343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:19.808214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 176/359] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:19.808990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:19.808990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:19.809946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:19.833794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:19.835374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:19.907391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 184/360] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:19.908674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.944606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 177/361] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:19.946038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:19.950574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:20.025012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:20.059772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769b0b7-a093-4599-89ef-ef99a3fdb674 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:20.113360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97c3b2-63aa-4329-bc9d-3425f8ec1a0b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:20.118221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:20.118856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:20.119134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:20.119404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:20.120088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:20.121580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:20.123090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:20.134117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 185/362] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:20.135657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:20.160826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:20.161461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:20.161775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:20.162489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:20.163255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:20.163745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:20.165517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:20.176689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 178/363] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:20.177732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:20.293150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:20.318514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9897f02-cdc3-4dff-b641-e1a00b9ca979 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:20.348611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84192f-830b-4f5e-a013-e7e14e7bd114 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:20.357614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:20.377933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:20.378600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:20.378880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:20.379155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:20.380527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:20.382441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:20.384588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 186/364] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:20 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:20.385547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:20.399961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:20.400642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:20.400926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:20.401196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:20.534260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:20.535756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:20.596907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26d24-1d96-481c-97cd-2371a2f12c3f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:20.659304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:20.661469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:20.661469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:20.661469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:20.698777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:20.702155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 179/365] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:20.702155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:20.712775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:20.781895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:20.784166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:20.882704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21143f-d319-4328-9e84-1a440fb28597 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:20.936795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:20.937638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:20.937857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:20.938130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:20.956717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:20.963056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:20.964748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 180/366] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:20.967938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:20.970741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 187/367] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:20.970741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.046539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:21.087797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:21.141466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958164f6-1607-4843-b064-fb24a9e8acd8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.141654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e1d42-829a-4c3e-a63c-21bc525c045b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.193120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.194191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.194191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.194415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.195589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.196582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.197103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.197227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.197495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.197768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.198926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 181/368] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:20 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.199120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.199555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.200787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.203911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 188/369] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:20 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.205994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.349904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3071d0-9c93-40f0-b80a-122fc7e4b547 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.357707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686285-df56-4d58-a59a-afa65a782368 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.394209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.394745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.395133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.395411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.396084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:21.396408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.397967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.399643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:21.400799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:21.417954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 182/370] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.419083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.423065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.423980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.424544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.425016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.426510 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.428370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.430442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 189/371] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.431654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.579400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3fbc-0cdc-4115-8c0a-223efb9478e7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.596824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae7b99-2b87-4132-adf2-49cc8e22fb66 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.638849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.639617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.640240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.640631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.640773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.640897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.641026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.641150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.641621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=98d7fe77a62546939bec3adf10e3790f) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:21.642027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.642332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.643446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.643763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.645861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 183/372] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.646575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.657546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 190/373] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.659725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.740605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:21.769878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:21.795167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2620c-e68a-472b-9f84-3b41aea80d20 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.846805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4172022-9639-47df-b4f7-89257c8e2be7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:21.863733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fff1c2-d18f-45f1-ae6f-4ce19669b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.864689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fff1c2-d18f-45f1-ae6f-4ce19669b416 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.865144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fff1c2-d18f-45f1-ae6f-4ce19669b416 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.865579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fff1c2-d18f-45f1-ae6f-4ce19669b416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.907086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:21.908085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:21.909500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:21.910069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:21.912483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.914943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:21.918837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 191/374] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:21.920811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:21.968019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fff1c2-d18f-45f1-ae6f-4ce19669b416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:21.969312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fff1c2-d18f-45f1-ae6f-4ce19669b416 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:22.075394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:22.088677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08c068-7895-4b2f-af83-61c7e140b43b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:22.118011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:22.129565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 184/375] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:22.130860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be71d-b5f5-4d4d-91b7-6dcddc64126f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.131229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be71d-b5f5-4d4d-91b7-6dcddc64126f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.131583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be71d-b5f5-4d4d-91b7-6dcddc64126f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.136366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 185/376] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:22.136366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be71d-b5f5-4d4d-91b7-6dcddc64126f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:22.144160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:22.145278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.145834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.146380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.147793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:22.148646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:22.151463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:22.161787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 192/377] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:21 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:22.163284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:22.315693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be71d-b5f5-4d4d-91b7-6dcddc64126f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:22.319840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c0bdc-8110-4f72-a6eb-cf1c88d6cff0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:22.358430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:22.359060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.359313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.359730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.360462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:22.360836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:22.362543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d31b-8163-4055-8cc7-b5d0d1cc4cf6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:22.369311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 186/378] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:22 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:22.370037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:22.370752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.371054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.371544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.371868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.372389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.372896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] PATH_INFO: `/v3/projects/130b40029b174542ac4b0eacc3caf048` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.375569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] RBAC: Authorizing `identity:get_project(project_id=130b40029b174542ac4b0eacc3caf048)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:22.377377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:22.381005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 193/379] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:22 2022] GET /identity/v3/projects/130b40029b174542ac4b0eacc3caf048 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:22.392544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:22.396842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:22.398234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:22.470269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:22.560678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 187/380] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:22.588952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:22.685769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85eac3a-1210-423b-9627-94851a3fbb0b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:22.754962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:22.756423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.757268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.758015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] PATH_INFO: `/v3/groups/4475dec1e12345b2a7b538d2d210eae0` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.773896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] RBAC: Authorizing `identity:get_group(group_id=4475dec1e12345b2a7b538d2d210eae0)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:22.776493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:22.780938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 194/381] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:22 2022] GET /identity/v3/groups/4475dec1e12345b2a7b538d2d210eae0 => generated 229 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:22.788692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:22.952885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46d9a-523e-4543-983c-eb1ebdfd15f0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:22.991234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:22.991852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.992140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.992449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.993182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] RBAC: Adding query filter params (group.id=4475dec1e12345b2a7b538d2d210eae0, role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=130b40029b174542ac4b0eacc3caf048) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:22.993494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:22.994943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:23.001608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 188/382] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:22 2022] GET /identity/v3/role_assignments?group.id=4475dec1e12345b2a7b538d2d210eae0&scope.project.id=130b40029b174542ac4b0eacc3caf048&role.id=ac2134ae61074ce6b751deadce042c42 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:23.229162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:23.388563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488765cd-4092-4d53-8ed1-f786219a5cb9 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:23.408164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:23.441112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:23.441777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:23.442186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:23.442652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:23.465332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:23.472196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:23.473163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:23.473979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 195/383] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:23.481468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:23.517463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:23.601608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96345bb-3d77-4205-ac6b-baba67bb0bcd None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:23.640587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:23.641195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:23.641473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:23.641744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:23.642416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:23.642850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:23.644233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:23.661072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 189/384] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:23.667488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:23.803728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b122-b7b8-42d4-890b-8e2dbff0f8ed None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:23.848915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:23.849981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:23.850489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:23.850992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:23.852526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:23.854255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:23.860349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 196/385] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:23 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:23.862758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:24.011420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:24.032246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dcc72-f97f-4f57-9888-d3cf9f4d95b2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:24.081695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ced22b-e683-49e3-a12c-f1b2693b711c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:24.082256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ced22b-e683-49e3-a12c-f1b2693b711c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:24.082531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ced22b-e683-49e3-a12c-f1b2693b711c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:24.082799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ced22b-e683-49e3-a12c-f1b2693b711c None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:24.083983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ced22b-e683-49e3-a12c-f1b2693b711c None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:24.085473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ced22b-e683-49e3-a12c-f1b2693b711c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:24.087460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 190/386] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:23 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:24.096715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:24.202475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:24.274048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5858-12b5-4833-8aca-1da6dbf407f0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:24.323967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:24.324733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:24.325017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:24.325296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:24.327396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7fe77a62546939bec3adf10e3790f, group_id=2d0fbcd03bbc42e9a40de46fae732006, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:24.329016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:24.417284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45604e-a024-4717-945a-8b13a9181b04 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:24.417658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45604e-a024-4717-945a-8b13a9181b04 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:24.418160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45604e-a024-4717-945a-8b13a9181b04 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:24.420254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 191/387] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:24.427399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe07267-9f80-43ee-b6a2-86b46a3e4fbe None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:24.428027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe07267-9f80-43ee-b6a2-86b46a3e4fbe None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:24.428576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe07267-9f80-43ee-b6a2-86b46a3e4fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:24.428682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 197/388] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 00:15:24 2022] PUT /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:24.464868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1fe07267-9f80-43ee-b6a2-86b46a3e4fbe None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:24.466631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1fe07267-9f80-43ee-b6a2-86b46a3e4fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:24.598519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 192/389] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:25.014586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:25.148169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0da6a3-280e-465d-be9d-5c87cd5e86bc None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.148789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0da6a3-280e-465d-be9d-5c87cd5e86bc None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.148789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0da6a3-280e-465d-be9d-5c87cd5e86bc None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.152137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 193/390] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:25.158005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49923b-ca2c-4543-bceb-1cde1e41b653 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.158460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49923b-ca2c-4543-bceb-1cde1e41b653 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.158940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49923b-ca2c-4543-bceb-1cde1e41b653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.191745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6227899-eda3-43b1-a2fb-037bf93dd187 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:25.198923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7a49923b-ca2c-4543-bceb-1cde1e41b653 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:25.200899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7a49923b-ca2c-4543-bceb-1cde1e41b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:25.217149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:25.278595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:25.279247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.279596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.279961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.315209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:25.328748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:25.330183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:25.331629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:25.332563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 198/391] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:25.336111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:25.363904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 194/392] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:25.365371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ef419-8cd2-484b-b4f2-cb34bee51c8f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.365662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ef419-8cd2-484b-b4f2-cb34bee51c8f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.366004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ef419-8cd2-484b-b4f2-cb34bee51c8f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.367585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 195/393] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:25.388150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.388453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.388894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.429535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:25.430763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:25.450521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:25.529320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50380-0899-4cb7-9c87-75df6fb6928d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:25.606977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 196/394] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:25.618696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:25.619927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.620722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.621259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.622794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:25.624292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:25.627005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c14c30-450c-40f6-b7c5-755642525d52 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:25.660987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 199/395] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:25.670720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:25.891341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43babb-ba30-4c1c-a7ce-dc13bce0be31 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.891736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43babb-ba30-4c1c-a7ce-dc13bce0be31 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.892220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43babb-ba30-4c1c-a7ce-dc13bce0be31 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.894722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 200/396] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:25.897752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f9a61-dbe9-4b67-84e8-d1ce456bcd74 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:25.915640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.916237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.916772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.942003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:25.942901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.943181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.943450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.944972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:25.946444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:25.948211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 197/397] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:25 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:25.955522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:25.958524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:25.960716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:25.970497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.010073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.077201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 201/398] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:26.078087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.095650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785a164-0878-4bc1-a547-053933fa25dd None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.144623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.145411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.145783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.146086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.147489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:26.149043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:26.157477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 198/399] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:25 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:26.158469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.243180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baaaf30-5123-4f60-9de0-fe98bd472772 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.294449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.296012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.296467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.296910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.308839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.313878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:26.323622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:26.326431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:26.327311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:26.329251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 202/400] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:26.330768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.339357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec1f46-3d10-4d97-87df-0390635494b5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.384932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.386212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.386813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.387412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.401441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:26.403816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:26.408160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:26.409154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:26.409921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:26.410543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 199/401] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:26.411107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.491191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffcda8-599d-492f-a8c5-351d718d1985 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.530754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.531459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.531797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.532067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.534252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7fe77a62546939bec3adf10e3790f, group_id=2d0fbcd03bbc42e9a40de46fae732006, role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:26.535694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:26.544720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d6b9-2d23-4568-bffd-bcb18b852e8b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.580524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.581135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.581413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.581679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.582337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:26.582636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:26.584007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:26.589327 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.593630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 200/402] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:26.594425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.620878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 203/403] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 00:15:26 2022] PUT /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:26.624059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.725168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d34cc3-3fdd-4046-8f95-7426ecd3abcf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.729759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.756310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862a86a-b58d-4fab-9631-e71301fd5d14 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.770215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.772862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.773410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.773927 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.788374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:26.793692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:26.794705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 201/404] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:26.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.805209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:26.806144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:26.806253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:26.806472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:26.807137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] RBAC: Adding query filter params (name=member) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:26.807434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:26.810040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:26.821276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 204/405] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:26.822121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:26.889625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:26.953664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084869c-63be-4ae3-9275-0a985c3d8457 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:26.968061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b91772-90ef-405a-a4a6-4180c6d6bdbb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.001603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.002299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.002575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.002908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.004220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.005865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.012880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 202/406] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.014821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.019820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.020888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.021374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.021939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.022811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:27.023304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.025121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.036107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 205/407] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.037471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.040843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:27.147128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8de0c-dfb4-475d-8319-a75dbf56da22 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.154058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3defeb9-2d5c-48f9-b422-e8fdf4d7b3c7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.181829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:27.191693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.192989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.193531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.193932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.195300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.197016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.198312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c349e-e0af-4292-a8c7-f18a473c6da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.198661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 203/408] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:27 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.199255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c349e-e0af-4292-a8c7-f18a473c6da5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.199426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.199820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c349e-e0af-4292-a8c7-f18a473c6da5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.200093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c349e-e0af-4292-a8c7-f18a473c6da5 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.204775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c349e-e0af-4292-a8c7-f18a473c6da5 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.206238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c349e-e0af-4292-a8c7-f18a473c6da5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.208209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 206/409] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:27 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.209101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae98a7-74aa-4745-a9a2-8b5e53c1409e None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.209374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae98a7-74aa-4745-a9a2-8b5e53c1409e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.209643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae98a7-74aa-4745-a9a2-8b5e53c1409e None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.210984 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 207/410] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:27.211509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae98a7-74aa-4745-a9a2-8b5e53c1409e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.303583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:27.336554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c82043-303f-4baa-8a7f-e4225c434760 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.338359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae98a7-74aa-4745-a9a2-8b5e53c1409e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.379474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.380099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83067be-6b78-4200-8746-3e6f1bd10908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.380242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.380522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.380801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83067be-6b78-4200-8746-3e6f1bd10908 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.380896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.381085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83067be-6b78-4200-8746-3e6f1bd10908 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.381361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83067be-6b78-4200-8746-3e6f1bd10908 None admin] PATH_INFO: `/v3/groups/4475dec1e12345b2a7b538d2d210eae0` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.381958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.382489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83067be-6b78-4200-8746-3e6f1bd10908 None admin] RBAC: Authorizing `identity:get_group(group_id=4475dec1e12345b2a7b538d2d210eae0)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.383768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.384244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83067be-6b78-4200-8746-3e6f1bd10908 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.394453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 208/411] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:27 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.394453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.394681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 204/412] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:27 2022] GET /identity/v3/groups/4475dec1e12345b2a7b538d2d210eae0 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.394681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.394681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.394681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.414124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:27.414378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:27.486435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:27.496381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 205/413] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:27.496561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.526035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06535c-950a-4936-90e6-ff1a4b985f96 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.566573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.567312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.567958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.568346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.571056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a9e583e2f324455bb4f1fd9b75f60ed, group_id=2d0fbcd03bbc42e9a40de46fae732006, role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.572741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.639805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e6a21-dcb4-4374-b54a-224cd4c1d29d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.678306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.678939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.679269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.679539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] PATH_INFO: `/v3/projects/130b40029b174542ac4b0eacc3caf048` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.680748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] RBAC: Authorizing `identity:get_project(project_id=130b40029b174542ac4b0eacc3caf048)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.682192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.683881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 206/414] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:27 2022] GET /identity/v3/projects/130b40029b174542ac4b0eacc3caf048 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:27.684739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.690707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:27.697808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 209/415] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 00:15:27 2022] PUT /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:27.699402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:27.839915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071b0ad-c82c-4ea4-a92b-0dcbbcef9a44 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.842697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:27.883818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.885424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.885708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.886004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] PATH_INFO: `/v3/projects/130b40029b174542ac4b0eacc3caf048/groups/4475dec1e12345b2a7b538d2d210eae0/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.887950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] RBAC: Authorizing `identity:create_grant(project_id=130b40029b174542ac4b0eacc3caf048, group_id=4475dec1e12345b2a7b538d2d210eae0, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.890386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.898633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d1e7c-5e40-4a30-a605-d4522727490c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:27.939584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:27.940247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.940555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.940823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.943026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] RBAC: Authorizing `identity:create_grant(project_id=3a9e583e2f324455bb4f1fd9b75f60ed, group_id=2d0fbcd03bbc42e9a40de46fae732006, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:27.944501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:27.961364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 210/416] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 00:15:27 2022] PUT /identity/v3/projects/130b40029b174542ac4b0eacc3caf048/groups/4475dec1e12345b2a7b538d2d210eae0/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:27.962562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:28.022449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 207/417] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 00:15:27 2022] PUT /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed/groups/2d0fbcd03bbc42e9a40de46fae732006/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:28.039323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:28.094748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579cc6e-0505-4618-a458-7e159fdd6221 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:28.141254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:28.141928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:28.142278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:28.142556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:28.161174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:28.167020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:28.168111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:28.169104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 211/418] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:28.175186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:28.332659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ca1db-726c-4343-815f-15cf693bbabc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:28.364255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:28.378715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:28.379664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:28.380108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:28.380612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:28.382718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:28.382718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:28.383148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34439902-a5f0-4c6d-a9f2-e272c75d1bec None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:28.393496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 208/419] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:28.398840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:28.576371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f12ec0-ed56-4536-8659-e213f54446cf None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:28.599224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:28.620909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:28.621419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:28.621700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:28.621987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:28.699318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eb38e-09e8-4439-a250-69d0c6ebbdc9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:28.699641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eb38e-09e8-4439-a250-69d0c6ebbdc9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:28.699868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eb38e-09e8-4439-a250-69d0c6ebbdc9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:28.702104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 209/420] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:15:28.707900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:28.708188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:28.708514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:28.730856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:28.732287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:28.745220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:28.746708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:28.810349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 210/421] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 31 00:15:28.938089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 212/422] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:28.944673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:29.058912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79625d5-e67a-4b73-ab60-38e10ff7a9b2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:29.074497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:29.097187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:29.097941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:29.098290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:29.098626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:29.099830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:29.101405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:29.103464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 211/423] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:28 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:29.109686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:29.142156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:29.160687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:29.266433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eef8b-1a57-42df-8981-4d6076e8dd1b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:29.322094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:29.322981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:29.323430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:29.323862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:29.325276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:29.328416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:29.330244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 213/424] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:29 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:29.336741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:29.361113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb50a-f2b1-44ab-b64b-d91533dab219 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:29.426167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:29.427190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:29.427618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:29.428090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:29.442378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:29.448120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:29.449230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:29.450226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:29.453046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 212/425] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:29.474142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:29.543438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:29.544277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:29.555840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46d200-12bb-4aea-80cf-9405fd97f7ed None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:29.614312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:29.616277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:29.616740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:29.617033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:29.617825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=98d7fe77a62546939bec3adf10e3790f) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:29.618152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:29.619628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:29.642636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 214/426] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:29 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 611 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:29.661267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ffc10-01bf-4d70-a920-387f362d4438 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:29.711613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:29.712233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:29.712555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:29.712821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:29.713487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:29.713785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:29.715234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:29.727881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 213/427] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:29.733611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:29.900255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31f3c6-a3e4-4ea1-a9a7-af09ef8aa0c8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:29.942376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6425ed-a830-4951-9134-b63c41f08d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:29.943250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6425ed-a830-4951-9134-b63c41f08d7a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:29.943526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6425ed-a830-4951-9134-b63c41f08d7a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:29.943793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6425ed-a830-4951-9134-b63c41f08d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:29.966804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:30.032824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6425ed-a830-4951-9134-b63c41f08d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:30.034717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6425ed-a830-4951-9134-b63c41f08d7a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:30.167837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 215/428] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:30.176732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:30.360583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc1147-a408-4a74-8ab4-536c27a0105d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:30.364670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:30.370443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e92eeb-e1cd-4cb8-bdcb-0f581247423f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.370957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e92eeb-e1cd-4cb8-bdcb-0f581247423f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.371334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e92eeb-e1cd-4cb8-bdcb-0f581247423f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.374573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 216/429] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:30.386048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6add5f0-01d4-4e2b-98eb-36137d0862a4 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.386827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6add5f0-01d4-4e2b-98eb-36137d0862a4 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.387321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6add5f0-01d4-4e2b-98eb-36137d0862a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.415188 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c6add5f0-01d4-4e2b-98eb-36137d0862a4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:30.416823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c6add5f0-01d4-4e2b-98eb-36137d0862a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:30.431309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:30.431865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.432220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.433296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] PATH_INFO: `/v3/projects/98d7fe77a62546939bec3adf10e3790f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.435576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] RBAC: Authorizing `identity:get_project(project_id=98d7fe77a62546939bec3adf10e3790f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:30.437913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:30.440291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 214/430] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:30 2022] GET /identity/v3/projects/98d7fe77a62546939bec3adf10e3790f => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:30.448918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:30.504202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:30.547539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 217/431] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:30.549426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dd642-711e-47b1-8531-e963c56b5e5a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.550124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dd642-711e-47b1-8531-e963c56b5e5a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.551064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dd642-711e-47b1-8531-e963c56b5e5a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.553344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 218/432] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:30.558229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff486692-ab11-4b43-bc22-7327c7560990 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.558511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff486692-ab11-4b43-bc22-7327c7560990 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.558785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff486692-ab11-4b43-bc22-7327c7560990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.580487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ff486692-ab11-4b43-bc22-7327c7560990 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:30.581704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ff486692-ab11-4b43-bc22-7327c7560990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:30.608762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d66d21-32a6-4488-87ee-643723ac465b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:30.652669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:30.653608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.654060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.655156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.657625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:30.659256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:30.661431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 215/433] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:30 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:30.667442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:30.683004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 219/434] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:30.719028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:30.735632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536b4d-74a9-49f9-8e71-14ef7806d6d1 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.735918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536b4d-74a9-49f9-8e71-14ef7806d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.736194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c536b4d-74a9-49f9-8e71-14ef7806d6d1 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.737861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 220/435] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:30.743654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.743936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.744210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.765912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:30.767251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:30.788911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fef8cc-d2de-43b1-93ae-a48b14b71601 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:30.841429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:30.842968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.843831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.844305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.845295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=98d7fe77a62546939bec3adf10e3790f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:30.846091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:30.848277 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:30.864991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 216/436] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:30 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=98d7fe77a62546939bec3adf10e3790f&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:30.867505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:30.877180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 221/437] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:30.985615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00656b2e-8caf-4548-b154-17dd5395f6aa None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:30.988235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.006016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.031438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.032410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.032832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.033242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.051527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:31.057713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:31.059666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:31.061073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:31.062958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.069905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 217/438] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:31.071510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.158128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5bbcf-19d2-4ad8-b1fa-c7578031f974 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.212988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.213964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.214426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.214846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.230042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbea555-28b6-402f-851e-58d47c650617 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.234988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:31.245337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:31.246278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:15:31.247074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:31.247720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 222/439] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:31.250009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.280942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.281966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.282395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.282671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.283653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:31.283653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:31.285085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:31.305823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 218/440] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:31.306778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.340280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.368805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.371883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e0a1e-aed4-4e2b-b887-41c15c7c8202 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.407443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.408075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.408373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.408647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.421833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:31.426108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3c6b9-d8db-4ccc-b2a8-b630619b1981 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:31.426779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:31.427801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:31.428663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 223/441] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:31.429335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.471497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.472107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.472412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.472690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.473353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:31.473648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:31.475017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:31.485938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 219/442] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:31.486863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.561830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab15e64-6547-4239-ba7e-1866db7c320e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.583806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.597633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32589a6f-6804-4c4c-825b-c0cbe6fba4f9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.603422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.604031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.604310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.604694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.643065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.645153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.645760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.646038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.646309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.646974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] RBAC: Adding query filter params (name=member) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:31.647272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:31.648693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:31.657693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 220/443] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:31.658523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.689950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:31.691425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:31.770420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0270f2-0316-4a19-8b1d-01d984dd63ac None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.804602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 224/444] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:31.805533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.807708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.808310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.809352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.809352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.829689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.870643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.880678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:31.882149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:31.918629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1e3d5-40eb-4b8d-90d8-5dcd10d38806 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.955539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.956172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.956555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.956936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.001979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 221/445] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:32.003434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.070252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:32.073959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.075480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.157589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d812d0-e8e8-4c89-8190-bb9299b7474e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.175489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:32.220513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.221606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.222048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.222457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.224703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.227374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.230478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 222/446] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.231815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.262553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 225/447] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:32.263857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.418229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c482f-dc8f-4576-af5a-ac140c70c653 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.436848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e4a6-fc77-4a15-9829-5c4466eada4d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.441845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:32.469108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.469761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.470032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.470299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] PATH_INFO: `/v3/projects/130b40029b174542ac4b0eacc3caf048` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.471671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] RBAC: Authorizing `identity:get_project(project_id=130b40029b174542ac4b0eacc3caf048)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.473200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.474983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 223/448] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/projects/130b40029b174542ac4b0eacc3caf048 => generated 344 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.475754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.491336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.491988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.492261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.492561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.493799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.495494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.497282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 226/449] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.498197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.531492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:32.633825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38924b-8d0e-467e-9e83-ed3ca211b470 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.653651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba94b57-9480-4e0b-9f45-31564770a65e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.671539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.672092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.672399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.672677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] PATH_INFO: `/v3/groups/4475dec1e12345b2a7b538d2d210eae0` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.673841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] RBAC: Authorizing `identity:get_group(group_id=4475dec1e12345b2a7b538d2d210eae0)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.675308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.676935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 227/450] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/groups/4475dec1e12345b2a7b538d2d210eae0 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.677703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.696788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.697410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.697683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.697955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] PATH_INFO: `/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.699074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=3a9e583e2f324455bb4f1fd9b75f60ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.700558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.702589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 224/451] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/projects/3a9e583e2f324455bb4f1fd9b75f60ed => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.703217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.760909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:32.783103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:32.789125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7165-cc73-43a7-95fc-f08ef35a531e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.805428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06b1fe-0b25-47f0-b69e-4cf3b3095ea0 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:32.827171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.827771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.828226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.828318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.829035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:32.829338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.830766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.836474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 228/452] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.837255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:32.844206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:32.844881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:32.845388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:32.845503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:32.846311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] RBAC: Adding query filter params (group.id=4475dec1e12345b2a7b538d2d210eae0, role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=130b40029b174542ac4b0eacc3caf048) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:32.846621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:32.848144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:32.855096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 225/453] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/role_assignments?group.id=4475dec1e12345b2a7b538d2d210eae0&scope.project.id=130b40029b174542ac4b0eacc3caf048&role.id=ac2134ae61074ce6b751deadce042c42 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:32.979899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de60f-ad8b-4ed6-90b7-6a4a859af71c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:33.021074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4a251-3127-4545-92ff-4bc7174ab1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:33.021827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4a251-3127-4545-92ff-4bc7174ab1ea None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:33.022103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4a251-3127-4545-92ff-4bc7174ab1ea None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:33.022373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4a251-3127-4545-92ff-4bc7174ab1ea None admin] PATH_INFO: `/v3/groups/2d0fbcd03bbc42e9a40de46fae732006` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:33.024381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4a251-3127-4545-92ff-4bc7174ab1ea None admin] RBAC: Authorizing `identity:get_group(group_id=2d0fbcd03bbc42e9a40de46fae732006)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:33.025660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4a251-3127-4545-92ff-4bc7174ab1ea None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:33.027254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 229/454] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 00:15:32 2022] GET /identity/v3/groups/2d0fbcd03bbc42e9a40de46fae732006 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:33.034264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:33.146629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5abbbb-d67f-4552-a7a1-359932386d02 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:33.163461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:33.180941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:33.181545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:33.181822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:33.182085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:33.182828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] RBAC: Adding query filter params (group.id=2d0fbcd03bbc42e9a40de46fae732006, role.id=c326d7ede2124b9e9b3714569c80f57e, scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:33.183131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:33.184606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8cd72-4533-42f9-87ab-50b1968be6bc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:33.191161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 226/455] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 00:15:33 2022] GET /identity/v3/role_assignments?group.id=2d0fbcd03bbc42e9a40de46fae732006&scope.project.id=3a9e583e2f324455bb4f1fd9b75f60ed&role.id=c326d7ede2124b9e9b3714569c80f57e => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:36.164435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c44359-2faa-41ce-857a-47db447b1a02 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.165298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c44359-2faa-41ce-857a-47db447b1a02 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.165767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c44359-2faa-41ce-857a-47db447b1a02 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.167708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 230/456] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:36.173757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0725e-38bd-4345-9b6a-77109b97b9ca None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.174915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0725e-38bd-4345-9b6a-77109b97b9ca None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.175399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0725e-38bd-4345-9b6a-77109b97b9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.198192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-aff0725e-38bd-4345-9b6a-77109b97b9ca None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:36.199973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-aff0725e-38bd-4345-9b6a-77109b97b9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:36.298635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 227/457] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:36.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b05fb-2298-4953-870d-63a147830dc7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b05fb-2298-4953-870d-63a147830dc7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b05fb-2298-4953-870d-63a147830dc7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.436522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 231/458] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:36.439842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.440153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.440421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.465614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:36.467574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:36.535017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3168-9e73-451f-afa6-2bc8100ef5c5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.535295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3168-9e73-451f-afa6-2bc8100ef5c5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.535824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3168-9e73-451f-afa6-2bc8100ef5c5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.537518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 232/459] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:36.549156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a515aee-fa19-4a54-87bd-66f717dac6f5 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.550292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a515aee-fa19-4a54-87bd-66f717dac6f5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.550840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a515aee-fa19-4a54-87bd-66f717dac6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.575011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5a515aee-fa19-4a54-87bd-66f717dac6f5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:36.579031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5a515aee-fa19-4a54-87bd-66f717dac6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:36.598449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 228/460] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:36.632783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:36.687427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 233/461] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:36.695409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:36.766235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:36.859311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b141-9bc6-442b-8288-5132d290e362 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:36.907277 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:36.908492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.908983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.909557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.926551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:36.929929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:36.948271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651dba-6d4a-4787-abae-405b64ff40c4 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.948503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651dba-6d4a-4787-abae-405b64ff40c4 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.948782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651dba-6d4a-4787-abae-405b64ff40c4 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:36.950530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:36.951872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:36.952962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:36.952962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:36.952962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:36.953138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 234/462] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:36.954966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 229/463] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:36.959556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.959556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.959556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.961885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.005492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:37.008296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:37.131732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 235/464] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:37.133767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.164470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:37.174653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d941157-4c41-421d-91e4-75694911f0f8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.249155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.250273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.250852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.251319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.252173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:37.252697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:37.254525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:37.274998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 230/465] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:37.276641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.293580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:37.319852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123442b-014a-49e4-86ee-5b339ca5108c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.376522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.377655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.377935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.378483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.393779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:37.396468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:37.420261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:37.421605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:37.422863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:37.422863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:37.422863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:37.425221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 236/466] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:37.428458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.446339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceec7b-8d7e-4dd1-bc0b-3500753486e6 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.521782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.522351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.523023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.523023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.539217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:37.541066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:37.559929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:37.561727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:37.565817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:37.565817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:37.565817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:37.565817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 231/467] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:37.565817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.579430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:37.610470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978286df-74c7-4370-94ef-e231f92e1c62 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.672402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.673383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.673814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.674085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.674761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:37.681327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:37.690381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:37.719310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb3ce-414b-477b-80a3-f9ea6f274d43 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.772757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.773345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.773657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.773974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.774631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:37.774941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:37.776504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:37.799606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 232/468] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:37.800370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.880860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:37.915738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71caf97-5583-433c-898c-4ea9d1d5a1c2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.946773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 237/469] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:15:37 2022] POST /identity/v3/users => generated 246 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:37.947619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:37.952509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.953178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.953455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.953722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.967835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:38.006204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:38.007076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 233/470] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:38.007929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:38.120067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9162d-e22e-4a7a-b011-c631cdabec19 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:38.127975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:38.128602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd5da-f9f7-4384-817d-adae3a961b60 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:38.128929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.129310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.129707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.130558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:38.131773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:38.132526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c1057-52fb-4197-8249-45e246dfb46e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:38.154680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 238/471] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:38.156767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c81d9-5ef2-4d34-84a8-051d97df6ba7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.156767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c81d9-5ef2-4d34-84a8-051d97df6ba7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.156767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c81d9-5ef2-4d34-84a8-051d97df6ba7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.158130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 239/472] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:38.158887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c81d9-5ef2-4d34-84a8-051d97df6ba7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:38.188406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:38.189070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.189359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.189625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.190314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:38.193553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:38.198563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:38.236413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:38.289934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c81d9-5ef2-4d34-84a8-051d97df6ba7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:38.333545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:38.334362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.334757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.360579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.360579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:38.360579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:38.360579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bc071-eb69-4112-abef-45c323cf3a32 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:38.360579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 240/473] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:38.360579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.361110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.361110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.376796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:38.377916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:38.449212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 241/474] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:38.451341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:38.456883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 234/475] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:15:37 2022] POST /identity/v3/users => generated 248 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:38.457996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:38.576185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:38.658578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c675f-2ebb-4fda-bd33-0a33e5478d56 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:38.672706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dcc7-1075-475c-9ecd-d6b1f4bac636 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:38.715818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:38.716643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.717193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.718059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.718843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:38.721556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:38.723486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:38.724413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:38.724891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:38.725356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:38.726212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:38.728079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:38.834953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 235/476] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:15:38 2022] POST /identity/v3/users => generated 249 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:38.835604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:38.863455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 242/477] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:15:38 2022] POST /identity/v3/users => generated 248 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:38.882350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:38.992157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c8c0-b4e3-43fc-a213-2aa68dfc1e75 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:39.050858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:39.051485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:39.051760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:39.052041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:39.052726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:39.054156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:39.129291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.sql.core [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 00:15:39.129291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 00:15:39.129291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [parameters: {'id': '4500ee1774844da381a59d3106715937', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 00:15:39.129291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88218) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result.read() Aug 31 00:15:39.149864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:15:39.150797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application yield resource Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise self.value Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application session.commit() Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 00:15:39.152009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.session.flush() Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application compat.raise_( Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 00:15:39.152907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:15:39.153789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result.read() Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:15:39.154632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [parameters: {'id': '4500ee1774844da381a59d3106715937', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:39.156455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 00:15:39.166503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:39.167529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 236/478] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 00:15:38 2022] POST /identity/v3/roles => generated 144 bytes in 316 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:39.167605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:39.290498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab57c8d-6252-47cc-a346-fbc529644966 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:39.292130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:39.330814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5d745-0559-48af-89ac-5731ac09787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:39.331809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d745-0559-48af-89ac-5731ac09787d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:39.332609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d745-0559-48af-89ac-5731ac09787d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:39.332992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d745-0559-48af-89ac-5731ac09787d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:39.348839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5d745-0559-48af-89ac-5731ac09787d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-bbf5d745-0559-48af-89ac-5731ac09787d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:39.355080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:39.356195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:15:39.358014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:39.364390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 243/479] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:15:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:39.364993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:39.476511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:39.490941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffda68e-565c-4df0-942c-8359fb8c72f0 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:39.529041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:39.529877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:39.530239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:39.530586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:39.531883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:39.532243 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:39.533844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff2f8e-119b-4b73-a4ef-b276ad72c3ae None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:39.546175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 237/480] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:40.277091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d477-a5ea-43af-88fc-303b72516d0f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:40.277475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d477-a5ea-43af-88fc-303b72516d0f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:40.277686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d477-a5ea-43af-88fc-303b72516d0f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:40.279517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 244/481] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:40.285150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be43cd0-51b9-49e9-9dce-d59bca8b6cfe None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:40.285433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be43cd0-51b9-49e9-9dce-d59bca8b6cfe None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:40.285688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be43cd0-51b9-49e9-9dce-d59bca8b6cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:40.310694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9be43cd0-51b9-49e9-9dce-d59bca8b6cfe None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:40.312595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9be43cd0-51b9-49e9-9dce-d59bca8b6cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:40.425777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 238/482] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:40.938013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845083d-dc36-4de4-91c6-75340b95ef05 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:40.938712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845083d-dc36-4de4-91c6-75340b95ef05 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:40.938712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845083d-dc36-4de4-91c6-75340b95ef05 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:40.940217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 245/483] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:40.950003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:40.950333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:40.950552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:40.988211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:40.989481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:40.994685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845083d-dc36-4de4-91c6-75340b95ef05 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:41.102597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 239/484] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:41.118651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845083d-dc36-4de4-91c6-75340b95ef05 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:41.156846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:41.157481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.157750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.158011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.169427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:41.175371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:41.177874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:41.179444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 246/485] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:41.189501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:41.222315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:41.336189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bed192-0e4a-4b76-bf6e-4b1d46aa67d2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:41.390157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:41.390773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.391049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.391316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.391991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:41.392292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:41.393700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48792290-a195-4b7b-8096-df9a7a88b6be None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:41.408258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 240/486] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:41.413652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:41.455878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:41.474762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239cf3-6836-4a8a-82c6-8bc5b08615a6 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.475268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239cf3-6836-4a8a-82c6-8bc5b08615a6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.475376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239cf3-6836-4a8a-82c6-8bc5b08615a6 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.476875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 241/487] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:15:41.482978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.483267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.483533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.518736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:41.519946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:41.571479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c253cd0-538f-4068-97be-c83d1793a64e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:41.618670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:41.620380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.620887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.621630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.637260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 242/488] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:41.637260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:41.647494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:41.649067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:41.652881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-5861523b-31da-4dee-ad21-b4f1942d00ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:41.669094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:41.671077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:41.672035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:41.672035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:41.672035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:41.672035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 247/489] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:41.672721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca789-b9de-4f4d-9d73-be8e1b4c1f33 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.672721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca789-b9de-4f4d-9d73-be8e1b4c1f33 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.672873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca789-b9de-4f4d-9d73-be8e1b4c1f33 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.678143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 248/490] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:41.678936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca789-b9de-4f4d-9d73-be8e1b4c1f33 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:41.780977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:41.785822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db78bc-f569-4324-a010-e6cefd050599 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:41.822775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca789-b9de-4f4d-9d73-be8e1b4c1f33 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:41.828830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a836e4f-2f58-4322-bf82-3c5818924cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:41.829525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a836e4f-2f58-4322-bf82-3c5818924cfb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.829795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a836e4f-2f58-4322-bf82-3c5818924cfb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.830062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a836e4f-2f58-4322-bf82-3c5818924cfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.843662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a836e4f-2f58-4322-bf82-3c5818924cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-5a836e4f-2f58-4322-bf82-3c5818924cfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:41.849970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:41.850912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:41.851925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:41.856487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 243/491] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:41.859772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.860077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.860394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.885211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:41.885830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:41.886106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:41.886377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:41.887086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:41.887317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:41.887418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:41.889343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:41.889896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:41.907127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 249/492] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:41.908051 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:41.986909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.000395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 244/493] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:42.000395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.046556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e4d95-be6f-4a7b-b077-ee6c1be57764 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.087821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.088494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.088776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.089094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.089768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:42.090067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.091462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.101366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 250/494] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:42.102237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.108771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.145120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d6abe-c764-4f19-b54a-3bfb6ca19894 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.183425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.184066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.184373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.184673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.195498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.197507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:42.213955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:42.214918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:42.215888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 245/495] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:42.217253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.239166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacc4e9-7860-4ade-ad03-bc06bf60d391 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.285805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.287262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.288217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.288662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.289100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.303772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:42.308035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:42.310215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 251/496] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:42.311098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.337774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf254fcb-193c-4653-9e20-2e82f816bb17 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.376786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.377409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.377684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.377951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.378045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.390027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.391512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:42.407041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:42.408347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:42.409231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:42.409231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:42.409231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:42.413406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 246/497] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:42.414573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.440983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43522ca0-3944-4b35-bea3-5c7970ea8228 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.483268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.484053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.484543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.485550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.486218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:42.486705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.489088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.537891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549d48-da21-474d-98fa-bfae12aac10f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.538989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 252/498] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:42.539837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.549923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.573891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.574441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.574717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.574986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.586669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:42.593875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:42.594845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:42.595706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 247/499] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:42.596403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.646268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b96f-d59c-4c4e-8b73-159e89dd6de4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.663642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.686012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.687714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.687714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.687714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.687950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:42.688410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.689827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1507bc0-ab04-4efc-8ee1-13d858800a9d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.698908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 253/500] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:42.699981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd1657-cb13-45d5-a73c-7f4aec57facd None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.700208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd1657-cb13-45d5-a73c-7f4aec57facd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.701103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd1657-cb13-45d5-a73c-7f4aec57facd None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.702690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 254/501] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:42.703409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd1657-cb13-45d5-a73c-7f4aec57facd None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.709276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471ba7c-12ec-4fe6-9457-39b68a6686db None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.749242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.749963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.750326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.750680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.751489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:42.751922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.753575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.768367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 248/502] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:42.769240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.807767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.826163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd1657-cb13-45d5-a73c-7f4aec57facd None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.870338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150070f-8a15-42b6-98a2-3f6588edd3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.870979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150070f-8a15-42b6-98a2-3f6588edd3ee None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.871261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150070f-8a15-42b6-98a2-3f6588edd3ee None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.871542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150070f-8a15-42b6-98a2-3f6588edd3ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.889953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc106-a19e-4d9f-b0dc-1462f442e6a1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:42.929871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:42.932613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:42.933405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:42.933545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:42.933994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:42.934531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:42.934832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.937037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:42.946743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 249/503] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:42.947612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:42.961028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150070f-8a15-42b6-98a2-3f6588edd3ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:42.962591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150070f-8a15-42b6-98a2-3f6588edd3ee None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.065559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.075101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958baeba-5f4c-4ac6-8b22-1cd0439c5d8b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.078641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 255/504] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:43.078641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.079617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.079617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.100996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:43.102233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:43.116620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.117253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.117537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.117809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.129129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.130598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:43.145517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:43.146458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:43.147524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:43.147524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.147524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:43.147524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 250/505] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:43.147524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.188353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 256/506] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:43.188353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.188353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.287516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc9cbf-525d-4b77-9b26-3c26832ae8a9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.311706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56475a4d-741e-4aff-a2b3-3c5244091284 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.323534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.324165 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.324473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.324744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] PATH_INFO: `/v3/users/glance` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.334557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.336119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:43.349299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:43.353093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 251/507] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:43.354208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.358126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.359086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.359728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.360014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.360281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.371508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.373139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:43.389899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:43.391070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:43.392584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:43.392584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.392584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:43.392584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 257/508] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:43.394136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.462940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.475232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59127a7a-83bd-41ee-9f89-62b0a4ed6adf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.508311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748fc91-f33b-4a57-b1cc-345cd5701bc4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.513859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.514805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.515088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.515358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.525995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.527463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:43.542634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:43.544191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:43.545278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:43.545278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.545278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:43.545278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 252/509] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:43.545278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.550782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.551399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.551675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.551946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.552629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:43.552934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.554314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.567861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 258/510] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:43.568735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.612812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.658877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d2d8a-a59b-4a74-ab14-98c86c5627e9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.683198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af40237-591d-4da3-8283-504b824dc399 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.694794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.695390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.695664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.696100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.696742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:43.697048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.698706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.703596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.720299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.721042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.721404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.721771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.722525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:43.722902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.724428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.737186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 259/511] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:43.737858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.738561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 253/512] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:43.739386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.848439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b7d9-b916-4cb1-a54a-2b3c625f846c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.848788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17593a-3b5a-4317-8f74-603d865ee0cc None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:43.863807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:43.886587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.887196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.887467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.887730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.888497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:43.888635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:43.888798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.889189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:43.889470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:43.889739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:43.890258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.902021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:43.906721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 254/513] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:43.906721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.906974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:43.919956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:43.921496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:43.922494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:43.922494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:43.922494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:43.922494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 260/514] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:43.922494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:43.954439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.009767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf66cf7-e190-491b-92a0-4f1f6c960a4b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.026788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95052053-13bf-4b35-81a7-cc987148bdfb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.043672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.045233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.045661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.045935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.054156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.055637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.062744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.063349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.063634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.063908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:44.068188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:44.069086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 255/515] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:44.075128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.109073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.137018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.138632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.170326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.183079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ca5b0-8880-4a44-b18d-7bfeeb1878cf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.219554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.220179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.220505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.220768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.228548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.230013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:44.242012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:44.243881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 256/516] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:44.245022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.253925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 261/517] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:44.254781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.342473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.348888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ec6e7-4e6e-4b5a-8ee2-3b634d31300a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.362367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9cbb9-5d1f-4146-9adc-44566b024635 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.383834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.384478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.384755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.385021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.391499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.392725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.394288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.397214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.397816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.398094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.398393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.399062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:44.399387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.400814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:44.407003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:44.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:44.409091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 257/518] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:44.410639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.418266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 262/519] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:44.418266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.518325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289fee-1ba6-42aa-b45d-cce9361955a6 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.521200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d567f3-2845-45e0-bf42-c2a931e08188 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.555790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.556453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.556928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.556928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.557083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.557161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.557248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.557410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.557638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:44.557938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.558132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:44.558435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.559370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.560418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.578678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 258/520] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:44.583683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 263/521] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:44.583860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.583954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.603842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.632886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.683736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87569-5e6c-4e65-a39b-06843f81cd1e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.700532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be369d-4aa5-4265-8ad3-e8b4309ae42e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.719439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.720036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.720313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.720673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.721359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:44.721659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.723138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.735204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.735841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.736121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.736422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.742507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 264/522] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:44.743325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.748465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.749933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:44.767395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:44.768589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:44.769630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:44.769630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:44.769630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:44.772468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 259/523] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:44.773615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:44.843544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.857027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f01a-1be6-4a60-a913-54b48d01748e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.866893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:44.886513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc49d6d-9b66-4a89-aff2-bad3c103d8db None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:44.894102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.895640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.896014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.896487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:44.897995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:44.900008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:44.922609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:44.925247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:44.925247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:44.925247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.004356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 265/524] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:15:44 2022] POST /identity/v3/users => generated 247 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:15:45.004356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.004715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.004715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.071960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.119211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 260/525] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:45.120116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.146274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a419bc2-e601-4b83-b855-28207144227c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.194279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.194900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.195182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.195749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.234281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.249824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae82be-9d47-492c-a607-d30265fd68fa None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.277604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.279581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.298869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.299623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.299734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.300009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.300839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=f1427404350643b2892469b0da65b788) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:45.301151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.302756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.309406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 261/526] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/role_assignments?user.id=f1427404350643b2892469b0da65b788&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:45.310479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.358810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.421084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 266/527] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:45.422391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.467866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd1422-bb56-4e39-ad0a-b484cdc3f7a3 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.505887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.522514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.524544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.525359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.526462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.527934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:45.529495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.533309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.549731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 262/528] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:45.550748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.559045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac211c20-206c-4213-9f98-8c1be958496f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.615633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.616953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.617890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.618633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.634471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.636261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.655757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:45.660197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:45.661285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:45.662282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:45.662282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:45.662282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:45.668578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 267/529] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:45.668578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.682583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4213a1e-d208-4eae-ac7e-1f796c758cba None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.735473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.737017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.737584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.738045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.755826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.758447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:45.781605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:45.782749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:45.783775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:45.783775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:45.783775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:45.785749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 263/530] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:45.786776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.801123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135e1f7-53b8-4bd9-9904-bf6dd62f565f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.826465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.846449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.847149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.847474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.847758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.855652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.858363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:45.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:45.872995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:45.874089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:45.879845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 268/531] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:45.881377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.916285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd297cfb-b388-471d-aed0-0d0aca07b026 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:45.954452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:45.955300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:45.955813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:45.956226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:45.957300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:45.957824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:45.959803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:45.971038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:45.980126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 264/532] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:45.981329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:45.995835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadd441-24f8-4167-b5e0-e905b5f76131 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.034474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.035149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.035679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.035838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.036667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:46.037047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.038587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.052667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 269/533] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:46.053908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.093045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:46.109433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0c2a4-5889-4f49-bcda-f67e87d6bcdf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.151747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.152472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.152758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.153106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.154152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:46.154459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.155900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.170509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8c8fe-75ff-4306-8822-32cb041e157e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.178421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 265/534] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:46.179678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.217119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.217919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.218302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.218674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.226417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.228342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:46.241294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:46.242985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:46.245099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 270/535] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:46.246067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.254559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:46.306503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694324e-88bd-442b-a003-c98e6fce4e34 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.356196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.358037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:46.359124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.360860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.361499 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.370897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f15c5-d0ab-479c-ab06-0032b5c8eb73 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.376521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.378978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:46.408192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:46.409749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:46.411286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:46.421371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 266/536] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:46.422913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.427089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.427892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.428490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.428873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.430954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=ad5664c09ed44105a1083cc90b3b739a) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:46.430954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.431875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.441730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 271/537] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/role_assignments?user.id=ad5664c09ed44105a1083cc90b3b739a&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:46.443322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.624295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf4a0-bda4-47ef-81a4-67489cc67817 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.680573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.683008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.683291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.683566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.684352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:46.685200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.687133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.718181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 272/538] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:46.718780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d750dc-915f-4978-958c-4f4492e204ab None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.732441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.746807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:46.765898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:46.766788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:46.767069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:46.767868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:46.768705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:46.769017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:46.770835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:46.780239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:46.790916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 267/539] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:46.797398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:46.955438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ca0d0-afe0-475c-bcf7-3b104ecbdf28 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:46.982378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ecdf9-3de4-4d5c-b5d9-310f2ea60965 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:47.006354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:47.006981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.007258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.007531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.008425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=b5f5987c024d44acbc7bf1502f11848f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:47.008739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:47.010565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f0c3-0944-4dbb-8aaa-946df7bbe85a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:47.017912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 268/540] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/role_assignments?user.id=b5f5987c024d44acbc7bf1502f11848f&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:47.066227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:47.066861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.067391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.067657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.068432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=a765c1f74578435d8f29fbf959e5f087) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:47.068743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:47.070179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c8836-5ea8-4800-bb8a-b4c2624d7fb1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:47.081409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 273/541] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:46 2022] GET /identity/v3/role_assignments?user.id=a765c1f74578435d8f29fbf959e5f087&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:47.177574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfd623-3ab9-4da8-91c9-625cf3186c82 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.177923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfd623-3ab9-4da8-91c9-625cf3186c82 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.178127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfd623-3ab9-4da8-91c9-625cf3186c82 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.179956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 269/542] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:47.186439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3861-d06d-4330-ab74-f44399e92db0 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.187006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3861-d06d-4330-ab74-f44399e92db0 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.187006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3861-d06d-4330-ab74-f44399e92db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.210194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fa5d3861-d06d-4330-ab74-f44399e92db0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:47.211345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fa5d3861-d06d-4330-ab74-f44399e92db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:47.313035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 274/543] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:47.494410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45968d35-1607-41b6-b534-d35afc8f42e3 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.494726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45968d35-1607-41b6-b534-d35afc8f42e3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.494960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45968d35-1607-41b6-b534-d35afc8f42e3 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.496621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 270/544] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:47.502009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.502286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.502606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.522365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:47.523714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:47.535808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:47.614898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 275/545] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:47.674952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45968d35-1607-41b6-b534-d35afc8f42e3 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:47.742833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:47.816102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45968d35-1607-41b6-b534-d35afc8f42e3 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:47.869082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:47.869730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:47.870005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:47.870273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:47.887388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:47.894609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:47.895595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:47.896432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 271/546] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:47.905481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.043278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4178e76-455e-433b-bbf7-9156b9a305c8 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:48.082471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.093823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:48.094715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:48.095186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:48.095629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:48.096635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:48.097320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:48.099540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:48.111385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 276/547] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:48.117791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.136735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:48.295201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2da1a-bb41-4e3e-82f8-224d27b2f58a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:48.339743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd847f-973a-418f-a5c8-67482b0a3b24 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:48.348657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:48.364515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:48.365157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:48.365429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:48.365697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:48.391544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:48.393043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:48.421077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:48.421705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:48.421976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:48.422242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:48.426316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:48.427885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:48.428857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:48.428857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:48.428857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:48.434256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 277/548] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:48.439412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.446562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:48.452848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:48.453943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:48.454747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:48.464206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 272/549] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:48.468739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.556863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:48.620845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b0017-29ef-47df-ae22-6c5b0ae2e9d5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:48.663135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:48.663751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:48.664026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:48.664293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:48.664996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:48.665293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:48.672633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:48.696954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 278/550] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:48.701489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.747336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d151b6-e439-4738-bc93-ea7d32f72cad None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:48.832583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:48.834417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:48.836084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:48.836397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:48.841358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:48.841358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:48.841358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:48.849426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 273/551] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:48.855573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:48.896590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:48.902242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e71bf0-463b-431f-911a-54e17cf82036 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:48.953068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:49.011091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:49.011750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.018321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.018679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] PATH_INFO: `/v3/users/swift` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.027859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e784188-9b63-4359-8385-8505396bc854 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:49.042070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:49.043615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1d623280-38cb-458c-a584-bebb75c2999d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:49.064705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:49.065769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:15:49.066684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:49.067452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 279/552] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:15:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:49.068928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe85a-acb8-4ca7-8974-da24e10b5522 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.069214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe85a-acb8-4ca7-8974-da24e10b5522 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.069484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe85a-acb8-4ca7-8974-da24e10b5522 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.077043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 280/553] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:49.078219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fe85a-acb8-4ca7-8974-da24e10b5522 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:49.122069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:49.123816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.124295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.124811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.148729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:49.151441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-51fd5808-0c40-4719-afab-63bfe69530c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:49.175995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:49.177110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:49.177948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:49.177948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:49.177948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:49.185635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 274/554] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:49.187817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.188380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.188853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.211786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:49.213444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:49.316832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 275/555] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:49.318566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:49.328650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fe85a-acb8-4ca7-8974-da24e10b5522 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:49.350993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:49.413221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:49.414159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.414613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.415027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.415922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:49.416412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:49.421890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcea69-6fe3-4eec-ae56-2fc8d6f13577 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:49.499777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 281/556] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 00:15:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:49.500881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10508a-4b96-442c-8948-f8b055171a16 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.501228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10508a-4b96-442c-8948-f8b055171a16 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.501476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10508a-4b96-442c-8948-f8b055171a16 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.506764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 282/557] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:49.507855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10508a-4b96-442c-8948-f8b055171a16 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:49.570740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:49.592618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51904d-1d0e-4449-bace-e18bdb48466b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:49.639172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:49.640120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.640429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.640709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.641376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:49.641733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:49.643118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2ba9d-2f8c-4093-8529-1543a76ad5aa None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:49.649569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10508a-4b96-442c-8948-f8b055171a16 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:49.656269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 276/558] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:49.657772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968a47-d67d-4f6c-909c-d9f498b76604 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.658069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968a47-d67d-4f6c-909c-d9f498b76604 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.658340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968a47-d67d-4f6c-909c-d9f498b76604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.682931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b9968a47-d67d-4f6c-909c-d9f498b76604 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:49.685678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b9968a47-d67d-4f6c-909c-d9f498b76604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:49.694587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:49.695210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.695720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.695868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.770652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:49.772161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:49.782526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:49.787898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 277/559] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:49.789210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8a30f-1ec4-4138-9394-961e3cf8d0f8 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.789492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8a30f-1ec4-4138-9394-961e3cf8d0f8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.789757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8a30f-1ec4-4138-9394-961e3cf8d0f8 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.791258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 278/560] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:49.791747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8a30f-1ec4-4138-9394-961e3cf8d0f8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:49.818860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:49.916255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 283/561] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:49.917627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:49.928863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8a30f-1ec4-4138-9394-961e3cf8d0f8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:49.971823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e67ff8-46cb-4058-8da9-01026c9f5455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:49.972755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67ff8-46cb-4058-8da9-01026c9f5455 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.973033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67ff8-46cb-4058-8da9-01026c9f5455 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.973377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67ff8-46cb-4058-8da9-01026c9f5455 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:49.987151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e67ff8-46cb-4058-8da9-01026c9f5455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-39e67ff8-46cb-4058-8da9-01026c9f5455 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:49.992175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:49.993381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:49.994233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 279/562] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:49.994997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:49.994997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:49.995835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.016823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:50.018106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:50.067236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b91349-a41c-4afd-a5d6-78a27fb7ae67 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.085576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.101777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 280/563] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:50.102645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.104941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.105712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.105988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.106254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.111082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.116911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.118562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:50.132834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:50.133891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 284/564] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:15:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:50.134812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.241868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386c4f9-ca3a-4263-8bf0-9e924a2d65a4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.259103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b78558-00ef-416e-a025-13c0248755f7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.282393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.283034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.283314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.283584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.295349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.296862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.312644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.313329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.313539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.313803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.314454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:50.314754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.316109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:50.320706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:50.321731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:50.322686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:50.322686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:50.322686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:50.322686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 281/565] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:50.323864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.335213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 285/566] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:50.335213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.391863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.402450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.447006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec9dc8-4ece-4908-b043-d76309342066 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.454133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196d790-b596-40c3-a151-07289ee53f42 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.482560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.483182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.483465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.483739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.484521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:50.484830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.486294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.491553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.492237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.492541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.492812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.504525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:50.506105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 286/567] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:50.506947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:50.509945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:50.511101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:50.511939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 282/568] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:50.513676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.615318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c01450-54d5-4758-8894-7dcf20bc62de None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.620377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.624575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054e9a4-3549-4df1-8256-5f98cd1c32ff None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.634127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.653121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.653963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.654244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.654525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.655160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:50.655461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.656880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.660502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.662790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.662790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.662790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.669709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 287/569] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:50.670607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.674525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.676016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:50.693047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:50.694306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:50.695395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:50.695395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:50.695395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:50.695395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 283/570] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:50.695642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.788392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cd6df-126d-4f2a-a262-aa97c246a65e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.805748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655c276-c013-42eb-8155-133f6411e0a4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.827662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.828294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.828605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.828877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.839735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:50.843209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:50.843837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:50.844116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:50.844424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:50.845653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:50.846615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 288/571] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:50.847555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.863398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:50.864920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:50.883587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:50.884869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:50.885844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:50.885844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:50.885844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:50.885844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 284/572] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:50.885844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:50.890790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.892145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.963840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0385a-6d29-4753-b894-7e5a8533c3d9 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:50.997902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f86c4-4574-495a-a0d3-51283dc3dafb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.000620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.001235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.001513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.001839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.002515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.002817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.004214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.012610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 289/573] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.014104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.039073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.039623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.039900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.040201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.047645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.049143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:51.061721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:51.062770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 285/574] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:51.063656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.129205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.131309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.134891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7c8eb-a5ce-44e8-8e48-5ef8b7174f48 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.170907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2925e1-b0ef-43bd-8685-b458cb23a29c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.175918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.176943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.177392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.177835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.178932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.179456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.181847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.196957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 290/575] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.198106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.209352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.209962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.210298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.210541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.211160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.211464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.212886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.222081 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 286/576] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.222925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.313528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c204aa-d15a-4419-b2c2-7add8011c5ff None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.327975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc558a3f-d0bc-4dde-b23e-d90f1ccbb534 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.351442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.352094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.352365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.352636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.353281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.353592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.354954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.355684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.366280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.366914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.367176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 291/577] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.367302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.367519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.368045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.383514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.385014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:51.400516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:51.401699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:51.403040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:51.403040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:51.403040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:51.403040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 287/578] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:51.403040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.434601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.521774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcfae9-5c86-4f4d-8fb7-905155dd3c4f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.522620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36891ea-63ed-415d-9eaa-cec5b1eb248b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.557819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.558447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.558725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.558997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.560281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.561196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.561472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.561744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.562440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.562743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.564242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.568042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.569527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.580916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 292/579] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.581765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:51.583891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:51.584860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:51.585797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 288/580] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:51.586364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.628318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.667906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.689006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67136be9-49a4-4af9-a97d-5ef7ecc40a17 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.690210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651cebf-b26f-49e4-a2d6-19091f217a94 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.725945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.726322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.726571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.726866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.726968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.727130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.727234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.727408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.734094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.735548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.738489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.739981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:51.746667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:51.747914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 289/581] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:51.749133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:51.758485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:51.759645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:51.761130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:51.761130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:51.761130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:51.761130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 293/582] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:51.761130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.852393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.856378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c7d1-bebd-459d-8c68-d84fbb0f2eff None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.871213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caceec-9b69-49c7-9241-e0f903a90aaf None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:51.891954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.892593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.892865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.893132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.893766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.894170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.895541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.900993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:51.907777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:51.930489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 290/583] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.930651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:51.930651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:51.930651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:51.930839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:51.930928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=1468fb64bc6d4b1899be323c0116c8e0) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:51.930928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:51.930928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:51.930928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 294/584] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/role_assignments?user.id=1468fb64bc6d4b1899be323c0116c8e0&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:51.930928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.048993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c7626-6833-4064-8a35-f62afa7adc87 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.050410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bea11-97dc-479e-beac-f722666a41cc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.115648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.116607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.116805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.117052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.117449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.118200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.118328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:52.118733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.118910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.119245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.120281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:52.120481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.120859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.122545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.139158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 295/585] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:52.141489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.150196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 291/586] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 00:15:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:52.150787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.180964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:52.219011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:52.281467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec309f9-9779-4669-8c8f-4053f4a7e569 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.305005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34470b-ad68-4ade-b065-5e6ab4047fce None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.329012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.329623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.329895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.330155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.330906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:52.331205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.332743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.337181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.337875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.338143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.338459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.346859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.350219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 296/587] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:52.350340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.350619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:52.382144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:52.383690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 292/588] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:52.384794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.486065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:52.500207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223be9ce-aebf-4409-9d1a-83ebd5deee8f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.515758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8139bc-853f-49b3-8f5d-a701019b86cb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.517011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:52.539087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.539744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.540024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.540298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/f1427404350643b2892469b0da65b788/roles/deb6dc5f801e411882108de755caff96` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.555350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.555881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=f1427404350643b2892469b0da65b788, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.555991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.556275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.556595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.557540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.569031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.570535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:52.586569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:52.593230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:52.594552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:52.594552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:52.594552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:52.594552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 293/589] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:52.594552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.634598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 297/590] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:52 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/f1427404350643b2892469b0da65b788/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:52.635346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.742724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfecb8-d4ec-432d-aedc-9ca55429b745 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.776628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:52.791448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.792759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.793493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.794668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.803722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b688dce-204f-47d2-ae14-1e714e8589c1 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:52.804060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.805594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:52.820556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:52.822790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 294/591] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:52.824839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.853330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:52.854329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:52.854860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:52.855353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:52.856313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:52.857539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:52.859619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:52.883445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 298/592] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:52.883445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:52.932939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:52.989104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170b1e1-9a3b-41ef-b792-ad564f271fba None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.010058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37dac-6573-4f4a-ae4f-8357bcebe144 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.042198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.043179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.043531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.043923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.045930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:53.045930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.047675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.068748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 295/593] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:53.068748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.096592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.098908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.099539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.100487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.102799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:53.103789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.108270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.131887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:53.155498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 299/594] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 00:15:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:53.155498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.210094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0221-dd7f-417e-af58-445597f5bdca None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.264415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.265119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.265549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.265973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.277761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:53.282758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.285170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.303564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d842b5-4df5-48ee-b48a-ed95c7c1ee14 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:53.304440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:53.305374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:53.306234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:53.306234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:53.306234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:53.306234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 296/595] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:53.307490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.356983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.357888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.358315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.358799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.367550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.369681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:53.385205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:53.386239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:53.387106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:53.392795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 300/596] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:53.398194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.468983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:53.473682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2ef6c-6542-4dd4-8102-7d578c1d8f84 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.517163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.517925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.518485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.518929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.531348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.532975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.537730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa042-1335-4e91-92b0-b7bdbc35bd2e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:53.551202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:53.552230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:53.553234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:53.553234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:53.553234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:53.556709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 297/597] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:53.558461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.588357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.589355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.589848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.590340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.592176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:53.592740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.594673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.609445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:53.617299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 301/598] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:53.618723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.716969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f317b1-67dc-497b-a968-85490656c224 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.781190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.783071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.783724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.784735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.786466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:53.786987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.789118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.816026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270ae09-d4da-494d-9a09-ed22725fc721 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.816608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 298/599] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:53.817658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.836308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:53.863783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.864831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.865680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.866348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.867776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:53.868614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.871187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:53.892111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 302/600] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:53.893365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:53.931207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519aca9-08ad-41c2-aa52-c4da14ef7ae9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:53.964781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:53.980891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:53.982096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:53.982519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:53.983695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:53.993281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:53.995169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:54.007747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:54.016753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 299/601] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:54.018269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:54.030767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c5ee-8429-4fc9-afbd-6d2645cde449 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:54.069846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd88f-6c53-40b7-a874-3fb1e2548252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:54.070579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd88f-6c53-40b7-a874-3fb1e2548252 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.070864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd88f-6c53-40b7-a874-3fb1e2548252 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.071544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd88f-6c53-40b7-a874-3fb1e2548252 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/ad5664c09ed44105a1083cc90b3b739a/roles/deb6dc5f801e411882108de755caff96` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.088948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd88f-6c53-40b7-a874-3fb1e2548252 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=ad5664c09ed44105a1083cc90b3b739a, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:54.090507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd88f-6c53-40b7-a874-3fb1e2548252 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.095284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:54.126101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2bde-387f-4b9f-bea0-0e7999fa9327 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:54.159768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:54.160544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.160903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.161166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.178786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 303/602] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:53 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/ad5664c09ed44105a1083cc90b3b739a/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:54.178786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c51d3-528a-4623-b701-21428391f347 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.178786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c51d3-528a-4623-b701-21428391f347 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.178786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c51d3-528a-4623-b701-21428391f347 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.179290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:54.179290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.179422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 304/603] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:54.180634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c51d3-528a-4623-b701-21428391f347 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-404849a4-fc1b-4090-b282-d3abfd01231b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:54.194216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:54.195437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:54.196313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 300/604] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:54.197289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.197613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.197891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.223948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:54.225194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:54.336602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 301/605] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:54.337593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:54.368499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c51d3-528a-4623-b701-21428391f347 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:54.389774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:54.420919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:54.422386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.422645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.423099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.424594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:54.425058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:54.427725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.456922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 305/606] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:54.459132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:54.464659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:54.529288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149f367-c569-4105-8676-7f16ec1271d5 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:54.588444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:54.589414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.589866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.590307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.592075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:54.592604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:54.595274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.619223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 302/607] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:54.629441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:54.694735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17a5a0-499a-43e9-8cdd-09043b365e0c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:54.743073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:54.744029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.744361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.744828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/b5f5987c024d44acbc7bf1502f11848f/roles/deb6dc5f801e411882108de755caff96` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.765516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=b5f5987c024d44acbc7bf1502f11848f, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:54.767455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.793394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473745a4-fe0f-4b98-8fbc-529489bc4819 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:54.819619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:54.831918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27b933-f33b-4d39-972e-6935ad9c1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:54.832870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b933-f33b-4d39-972e-6935ad9c1392 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.833312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b933-f33b-4d39-972e-6935ad9c1392 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.833773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b933-f33b-4d39-972e-6935ad9c1392 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/a765c1f74578435d8f29fbf959e5f087/roles/deb6dc5f801e411882108de755caff96` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.851935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27b933-f33b-4d39-972e-6935ad9c1392 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=a765c1f74578435d8f29fbf959e5f087, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:54.853361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27b933-f33b-4d39-972e-6935ad9c1392 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:54.889797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 306/608] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:54 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/b5f5987c024d44acbc7bf1502f11848f/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:15:54.896481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:54.925045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:54.960666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 303/609] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:54 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/a765c1f74578435d8f29fbf959e5f087/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:54.962394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580904a-87fc-4a71-9266-85d9ebe3e00a None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.962794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580904a-87fc-4a71-9266-85d9ebe3e00a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.963173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580904a-87fc-4a71-9266-85d9ebe3e00a None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:54.971138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 304/610] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:54.977508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:54.977722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:54.978060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:55.001489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:55.002878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:55.073555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442247e6-1946-4b54-adf7-ee1a55370384 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:55.102297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 305/611] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:15:55.123756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:55.125082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:55.125611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:55.126083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:55.142096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:55.147674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:55.148717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:55.149583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:55.156775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 307/612] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:55.161501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:55.297144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cd6a8-bf58-421d-b99b-c19b9df52891 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:55.298777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:55.345592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:55.346255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:55.346579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:55.346853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:55.347713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:55.348019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:55.349485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:55.365096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 306/613] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:55.370334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:55.482160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:55.523379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:55.562704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0d4fc-4eeb-4ac5-8d45-1d197c7154af None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:55.607304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d4ec9-20a3-4605-8f39-1bfea730523a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:55.641506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:55.642025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:55.642300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:55.642567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:55.662107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:55.662756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:55.663101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:55.663521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:55.667182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:55.669288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:55.691262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:55.697787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:55.698782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:55.699615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 307/614] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:55.706909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:55.709482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:55.711587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:55.711587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:55.711587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:55.714071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:55.716120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 308/615] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:55.720208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:55.741610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:55.832677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c444ee-c2bd-4dc3-99f0-62f7babbc34a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:55.876285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:55.876928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:55.877204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:55.877477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:55.878135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:55.878433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:55.879809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:55.904959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 308/616] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:55.910755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:55.947912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef417f-0811-4049-9336-05de8c341593 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.020250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:56.032690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.033312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.033596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.033967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.034730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:56.035113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.036532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.060779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 309/617] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:56.065664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:56.101983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:56.128357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91cb66-3832-4564-8241-6aa387adcca2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.201651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.202217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.202496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.202770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.225403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.226958 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:56.255395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:56.257635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:56.259649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:56.259649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:56.259649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:56.259649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 309/618] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:56.266315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:56.274628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9214ecd-042b-49d1-b935-867032543026 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.375444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.376093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.376405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.376668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.379625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32323b4a-c7b8-4a1b-a99e-6d1f984c1740 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.397195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.398840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.420609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.421228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.421505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.421774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.422422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:56.422936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:56.423043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:56.423925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:15:56.424950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:56.425506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.428476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 310/619] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:56.439829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:56.444836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:56.458376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 310/620] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:56.466069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:56.523776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:56.611496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b944-7ed7-44b2-be8c-2c6dc5f75702 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.670183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e13c67-cecd-41f1-94b4-50925d822127 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.681116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.681764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.682135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.682397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.683138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:56.683444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.685389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb92f4b-d09b-4720-800d-eb8313ee5444 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.709278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 311/621] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:56.710572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c8243-9952-4f90-9703-1867d6598540 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.710947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c8243-9952-4f90-9703-1867d6598540 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.711223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c8243-9952-4f90-9703-1867d6598540 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.712705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 312/622] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:56.715016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c8243-9952-4f90-9703-1867d6598540 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:56.721234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.721886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.722156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.722419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.730397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.731871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b7081d2a-2625-472c-b605-bec802441c4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:56.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:56.747508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:15:56.748960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:56.752628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 311/623] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:56.754041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.754320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.754592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.774671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:56.775906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:56.841383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:56.858259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 312/624] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:56.858907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:56.864658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c8243-9952-4f90-9703-1867d6598540 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:56.882236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:56.916221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:56.917057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:56.917337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:56.917608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:56.933890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:56.933890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:56.954645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:56.955874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:56.956949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:56.956949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:56.956949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:56.958017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 313/625] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:15:56.963296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.049890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be949c07-5f39-4088-94de-0bef2d0c749a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.088542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.089335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.092349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.092349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.092349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:57.092349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.092774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.112650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 313/626] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:57.125527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.170951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb859e8-2a99-4b03-ae12-71d71e1985a6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.200641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:57.220867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.221891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.222373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.223005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.224429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:57.224973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.227043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.247157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 314/627] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:15:57.248916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.273882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2773af9-9867-4736-b17f-be48c90cadfa None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.287873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:57.312119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41c950-fe56-49ca-a6f2-2b83465ee17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.312774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41c950-fe56-49ca-a6f2-2b83465ee17e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.313103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41c950-fe56-49ca-a6f2-2b83465ee17e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.313427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41c950-fe56-49ca-a6f2-2b83465ee17e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.376640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0eec33-caa2-481b-bc89-14b5b2191d22 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.412021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41c950-fe56-49ca-a6f2-2b83465ee17e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.414448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41c950-fe56-49ca-a6f2-2b83465ee17e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.425883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.426525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.427345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.427345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.439745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:57.444915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:57.446466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 315/628] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:57.447498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.477565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:57.536673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:57.537490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 314/629] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:15:57.538826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed09b9-3ca3-4670-b8de-05e3924fb879 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.539262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed09b9-3ca3-4670-b8de-05e3924fb879 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.539373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed09b9-3ca3-4670-b8de-05e3924fb879 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.544458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 315/630] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:57.544458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed09b9-3ca3-4670-b8de-05e3924fb879 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.556308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37bd88-efd8-4645-af11-93ed039df6e2 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.592883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.593526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.593807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.594080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.601555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.602668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-cbdf8269-5815-43a2-8aaa-1f9b491fb4ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:57.614474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:57.615743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:57.616657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 316/631] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:57.618028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80271db-eb98-44b0-b365-e3c8b5fb32e7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.618478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80271db-eb98-44b0-b365-e3c8b5fb32e7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.618758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80271db-eb98-44b0-b365-e3c8b5fb32e7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.621119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 317/632] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:57.621687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80271db-eb98-44b0-b365-e3c8b5fb32e7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.652349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed09b9-3ca3-4670-b8de-05e3924fb879 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.689286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.689891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.690164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.690636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.691096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:57.691395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.693055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556ff2-fdd9-4e02-9b4d-83067a6b215f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.702911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 316/633] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:57.704314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.704626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.704894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.723808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:57.725053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:57.727112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:57.731410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80271db-eb98-44b0-b365-e3c8b5fb32e7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.769116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.770074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.770507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.770778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.783269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.784762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.786643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:57.793099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 317/634] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:57.794134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-8a34d5d6-dd32-4f18-9ca9-4ee7ea66c73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:57.800464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:57.801815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:57.802826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:57.802826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:57.802826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:57.802826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 318/635] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:57.802826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.803122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.803276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.823345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:57.824576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:57.911071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 319/636] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:15:57.911071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:57.925087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1e1e8-77d5-4bef-8431-10adbb0f90df None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:57.975363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:57.976103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:57.976503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:57.976966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:57.977738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:57.978115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:57.979776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:57.992462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.003316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 318/637] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:58.004526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.057055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950922a-b402-4caf-8e34-0e820b80925b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.085843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.099452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.100097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.100407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.100680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.116157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.117948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:58.135666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:58.137011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:58.138278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:58.138278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:58.138278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:58.138489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64f07b-fee3-4c70-8925-666f239b29e2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.141019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 320/638] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:58.141638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.192058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.192703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.192985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.193255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.193905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:58.194210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.195600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.209091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 319/639] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:58.209933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.275551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ff76d-6816-4412-aaa1-4ed4aa43d31c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.276293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.312310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.313015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.313301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.313571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/1468fb64bc6d4b1899be323c0116c8e0/roles/deb6dc5f801e411882108de755caff96` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.321012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7a7e-6094-4f93-9d5c-6736e412e536 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.331198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=1468fb64bc6d4b1899be323c0116c8e0, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.332689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.360258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.361143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.361433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.361733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.372298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:58.377294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:58.378995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 320/640] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:58.380600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.422106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 321/641] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:15:58 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/1468fb64bc6d4b1899be323c0116c8e0/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:15:58.422952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.459968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.518348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14345b-d34c-4b6d-a2cf-260bd057f3bf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.543180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.559734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a6c5-d973-47cd-b274-fa8d32809405 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.564675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.565303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.565582 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.565854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.566508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:58.567111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.569070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.585678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 321/642] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:58.587048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.606849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.607496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.607771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.608045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.621457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:15:58.627649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:58.628776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:58.629632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 322/643] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:58.630330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.721544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c70b91-907c-4dae-a992-dd15f4a04071 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.736195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.755595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b1e6-2973-445a-af4f-2b480197843c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.766192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.766829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.767109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.767443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.777087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.778617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:15:58.791988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:58.793950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 322/644] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:58.795680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.800392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.801016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.801297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.801571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.802240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:58.802551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.803982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.814046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 323/645] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:58.814991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:58.856984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:58.932766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d79a7b-f707-4daf-a254-dc326cd2ab9b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.944287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bb766-6a3b-40ab-aacf-381f42dd6b2d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:58.973312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.974311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.974657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.975172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.984137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:58.984826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:58.985217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:58.985333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:58.985989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:58.986292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.986448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:58.987753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:58.988033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.000933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 324/646] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:59.000933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:15:59.003148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:59.004886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:15:59.006050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:59.008596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 323/647] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:59.009732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.016318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.124408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef8e64-fdec-42c1-a764-4ec06becbc7c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.138029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.141824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc0cf-1d55-4fdb-ba30-b7a8f722c178 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.165101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.166257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.166805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.167569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.168982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:59.169537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.171011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.181857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.182494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.182774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.183047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.189960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 325/648] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:59.189960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.205770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.207663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:59.227556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:59.228777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:59.229870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:59.229870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:59.229870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:15:59.231611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 324/649] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:59.232704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.272118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.319524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab75831-b6f4-4dff-8c10-82c89b8d58c8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.361353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055f00-615e-453a-af95-e29880c886ca None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.365295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.365937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.366212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.366482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.379670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.382302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.402025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:15:59.402306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:15:59.403937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:15:59.405214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:15:59.405214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:15:59.405214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:15:59.405214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 326/650] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:15:59.405214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.411337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.411337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.411337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.411841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:59.411841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.415815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.445673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 325/651] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:59.445673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.454456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.539201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d083e7-d5d9-4ad5-990f-59d5064cd2dc None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.574204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.585473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a571993-fcd9-4d7f-9d39-bc021efdcb18 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.590161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.591906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.592190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.592590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.593339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=f1427404350643b2892469b0da65b788) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:59.593651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.595104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.601790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 327/652] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/role_assignments?user.id=f1427404350643b2892469b0da65b788&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:59.602627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.648396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.649154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.649441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.649703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.650367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:59.650671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.652131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.672305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 326/653] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:59.673484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.751205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f5a0e-9df0-4732-9a31-7e1403d40c24 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.783926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.810690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.811546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.811814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.812620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.813047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:15:59.813153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.814615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:59.837035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6433f76-23ed-493f-986f-79c7865bbe57 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.845692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 328/654] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:15:59.845692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:59.881933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:59.882572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:59.882872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:59.884965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:59.915804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:59.977992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de0115-5b88-45c2-bdad-06db015aca1f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:59.980925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:59.983529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.020589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.021598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.021970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.022338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.031320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.033016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:00.053376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:00.054660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:16:00.055774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:00.058314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 329/655] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:00.061408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.092242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:00.141245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 327/656] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:16:00.142745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.211128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb65dfd-8409-486c-b6c5-f249cebf0a2b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:00.221660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:00.263607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.265774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.266481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.268428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.278765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.280734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.288811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128939b-f1b6-4f3b-99af-4371955e022b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:00.299648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:00.300910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 330/657] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:00.302046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.350839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.351750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.352032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.352304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.354186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:00.358297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.358297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.384694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 328/658] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:00.385696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.426132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:00.458080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d07bc-4310-4fff-a903-caf04872fb6e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:00.500932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.501764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.502084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.502346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.520338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.520338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:00.537838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:00.539683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:00.541397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:00.541397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:00.541397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:00.541397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 331/659] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:00.541397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.554574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b606f94-635f-41ee-883c-d1edf1c6a61d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:00.579226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:00.620268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.621540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.622141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.622688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.624097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:00.624722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.626460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5c0cd-7582-4d53-93bc-c95fc704f85f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.648063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 329/660] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:00.649970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c565a4-c744-4687-bcf3-5628d604dae9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.650412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c565a4-c744-4687-bcf3-5628d604dae9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.650837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c565a4-c744-4687-bcf3-5628d604dae9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.654169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 330/661] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:00.655894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c565a4-c744-4687-bcf3-5628d604dae9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.680929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63955e-1f14-4d88-886a-71220ac0438b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:00.730374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb936e-4187-4427-93ed-f4550c614db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.731176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb936e-4187-4427-93ed-f4550c614db8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.731575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb936e-4187-4427-93ed-f4550c614db8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.731917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb936e-4187-4427-93ed-f4550c614db8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.763293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:00.783257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c565a4-c744-4687-bcf3-5628d604dae9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:00.831081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb936e-4187-4427-93ed-f4550c614db8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.832603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb936e-4187-4427-93ed-f4550c614db8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.835279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:00.836158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.836517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.836851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:00.837636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:00.838001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:00.839470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:00.858106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 331/662] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:00.858818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:00.859006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:00.984828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 332/663] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:16:00.986846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:00.987318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:00.987774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.003289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df68a-9aa3-43bb-bdd4-b7e8eb1f10d7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.007826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:01.009053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:01.044893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.045467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.045750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.046024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.061934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.104509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 333/664] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:01.104509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.131851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.133498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.177744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.250845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9eb53-f998-4bcd-b4e8-279189a8248f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.252238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 332/665] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:16:01.253257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.299863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.301843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.301843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.302102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.305412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.312439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.313882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:01.326429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:01.327624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:01.328560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 334/666] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:01.329053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.373121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4fc6-746e-4d6e-aba1-66998c731a28 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.421936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.422574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.432362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.432362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.442643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.444133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.452072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d53f-b631-4feb-85d7-15a326aeb4cb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:01.460027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:01.461113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:01.462183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:01.462183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:01.462183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:01.462741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.463707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 333/667] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:01.464654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.490937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.492003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.492290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.493025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.504591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.506029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:01.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:01.525673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:01.526555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:01.526555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:01.526555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:01.531158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 335/668] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:01.532108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.584889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.591633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1c7ba-0b4c-416c-9f9c-0144f540a5ee None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.627389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.627998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.628278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.628600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.629279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:01.629583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.631025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.644293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 334/669] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:01.645164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.658085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8c3cb-41ff-4aac-8a1c-5183b38c1405 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.698030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.698708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.698945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.699216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.710967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:01.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:01.718654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:01.719493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 336/670] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:01.720180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.745437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.771681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ff85f-8216-48cb-9072-4f088b4b7fd6 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.816959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.817580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.817860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.818134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.818785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:01.819143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.820561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.833646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 335/671] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:01.834510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.841043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f871a7-5696-4fad-99e2-1ef9fde085c7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.856016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.879525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.880141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.880462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.880732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.881373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:01.881677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.883118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.895723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 337/672] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:01.896579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:01.943333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942210-3ac0-47ec-96be-4101c51bae9e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:01.978843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:01.979446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.979720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.979988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.981555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=ad5664c09ed44105a1083cc90b3b739a) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:01.981709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:01.982874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:01.983008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ad6ce-89d7-4ca5-9c82-2ae4b7543e09 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:01.989027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 336/673] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/role_assignments?user.id=ad5664c09ed44105a1083cc90b3b739a&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:01.989979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeb3a0-c664-4157-8bc3-bfd86003d24e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:01.990260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeb3a0-c664-4157-8bc3-bfd86003d24e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:01.990609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdeb3a0-c664-4157-8bc3-bfd86003d24e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:01.992052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 337/674] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:01.992800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdeb3a0-c664-4157-8bc3-bfd86003d24e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.005746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b983-4d9c-4e0d-af05-971e8cca82b7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.047374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.048152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.048471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.048761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.049486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:02.049817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.051328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.069309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 338/675] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:02.069309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.105153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:02.131924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdeb3a0-c664-4157-8bc3-bfd86003d24e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.173377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.174012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.174281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.174552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.183194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.185203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-d0d0e142-f34d-45b2-ae2f-3454c82c49ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:02.198722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:02.199898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:02.201948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:02.203630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b737543-17e4-4889-b424-65cdd7ffbaa0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.204482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 338/676] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:02.206599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.206977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.207448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.227721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:02.229051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:02.266398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.268358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.268358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.268358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.277459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.280099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:02.292738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:02.293895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:02.295538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:02.299547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 339/677] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:02.300348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.303228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:02.340747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 339/678] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:02.341657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.429266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:02.439244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117f70d-7efb-42f3-8236-b5ccc27c3e70 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.474100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8f487-c8f8-404e-9d1b-e0eee4807190 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.493885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.494805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.495362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.495840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.509974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.511782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.523932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.524656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.524998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.525353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.526882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:02.527457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.530050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:02.530750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:02.531679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:02.532633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:02.532633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:02.532633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:02.534770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 340/679] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:02.535853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.556415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 340/680] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:02.558153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.610236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:02.665667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee911-32b0-4f00-b0aa-3b637f94d21b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.700106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53a684-a3c9-4cba-acfc-698134ebf57b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.713979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.714607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.714886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.715176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.716678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:02.716980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.718425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.738913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 341/681] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:02.738913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.743995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:02.751255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.751868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.756693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.756693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.756693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:02.756693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.756693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.768843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 341/682] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:02.769940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.861757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f20ca-ec88-45ac-acb5-2168d3a4ff9e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.876701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:02.887134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684a460-2435-4c68-af7b-59ffe5fdc31e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:02.900526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.901286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.901611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.902010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.903868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=b5f5987c024d44acbc7bf1502f11848f) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:02.904493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:02.906912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:02.913520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 342/683] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/role_assignments?user.id=b5f5987c024d44acbc7bf1502f11848f&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:02.914330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:02.944459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:02.945443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:02.945808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:02.946257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:02.963436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:02.970145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:02.971544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:02.972575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:02.973637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 342/684] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:02.974575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:03.083643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:03.103515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd4afb-5ccb-44d4-87de-9ad0780dce30 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.153468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:03.154174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:03.154510 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:03.154950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:03.155503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=a765c1f74578435d8f29fbf959e5f087) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:03.155810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:03.157369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:03.165375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 343/685] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/role_assignments?user.id=a765c1f74578435d8f29fbf959e5f087&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:03.166280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:03.192793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47699ffd-b012-48bd-8d76-5fea21149c1a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.244499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:03.260798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:03.261832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:03.262871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:03.263347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:03.272440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:03.274373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:03.289768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:03.290793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:03.291741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:03.295917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 343/686] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:03.303739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98703355-c874-4417-b1b7-7844181fbf6c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.304454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:03.352195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:03.353273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:03.356448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:03.356448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:03.356448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:03.356448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:03.358679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:03.385549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 344/687] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:03.392798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:03.455080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c900f6-1759-4248-a32b-a1e7274d7c30 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.478290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:03.511840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06e9a9-8baf-46c8-81fd-3fa872a1935a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.522529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:03.526439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:03.528065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:03.529449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:03.530536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:03.532138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:03.533164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:03.536689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:03.560958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:03.561614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:03.562023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:03.562314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:03.577549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:03.580017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 344/688] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 00:16:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:03.585838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:03.590245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:03.603993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:03.605135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:03.606157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:03.606157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:03.606157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:03.612441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 345/689] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:03.614425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:03.802754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c6a17-63b2-4c7b-947f-3199e8dbb948 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.873828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb76dbf-592b-4cee-a8fb-7bfcfcb7f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:03.875205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb76dbf-592b-4cee-a8fb-7bfcfcb7f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:03.875847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb76dbf-592b-4cee-a8fb-7bfcfcb7f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:03.876453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb76dbf-592b-4cee-a8fb-7bfcfcb7f0d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:03.939803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f68b6e-8bed-4f81-9115-b16905370bf9 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:03.972997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:04.008289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb76dbf-592b-4cee-a8fb-7bfcfcb7f0d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.011838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb76dbf-592b-4cee-a8fb-7bfcfcb7f0d7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.065213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:04.066159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.066889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.067346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.068196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:04.068747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.070131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.101658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 346/690] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:04.109895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:04.176909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:04.200305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 345/691] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 00:16:04.202587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf675-b821-4b4c-be64-b6bec5059777 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.203249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf675-b821-4b4c-be64-b6bec5059777 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.203942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cf675-b821-4b4c-be64-b6bec5059777 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.206940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 346/692] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:04.209245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf675-b821-4b4c-be64-b6bec5059777 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:04.301757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b6fa-df81-4fec-806c-5f8de8a165b5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:04.337827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:04.351988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:04.354078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.354078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.354078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.363568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.365687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.381511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cf675-b821-4b4c-be64-b6bec5059777 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-716f474d-51c3-4514-bb43-fb6481b8c2c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:04.384065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:04.385047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:04.385975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:04.389107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 347/693] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:16:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:04.390581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.391136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.391611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.411441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:04.413143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:04.433364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:04.434526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.434916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.435401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.448997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.451962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:04.470555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:04.473307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:04.475212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:04.475212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:04.475212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:04.475212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 347/694] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:04.475212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:04.507195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 348/695] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:04.508515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:04.526683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:04.608968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35c6c-9cce-4e87-bce7-d8dfd4a7e2f9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:04.634718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:04.651726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:04.652370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.652709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.653571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.654664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:04.654979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.656478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.681765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 348/696] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:16:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:04.683579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:04.809900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e096e13-2e53-42cb-9364-7a13088ea9b6 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:04.832151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013a5e0-6a26-4bc3-a743-a339265c611a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:04.866564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:04.867112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.867443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.867717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.897267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:04.897950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:04.898239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:04.898507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:04.899280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:04.899586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.901128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.916526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 349/697] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:04.917402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:04.953238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:04.983428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:04.985314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:04.995415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:05.036654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec6a9e-cd99-47d8-a048-c4576e34fc13 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.081565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.082219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.082496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.082774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.097314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.099080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.110966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 349/698] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 00:16:05.112132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:05.127730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:05.128795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:05.129704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:05.129704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:05.129704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:05.134309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 350/699] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:05.135777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.204625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:05.241892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41118a5c-7906-4a7a-b7a8-b0afae65cf56 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.289175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ed0f9-bcc4-41af-9ca1-cd1afc46a095 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.305464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.306750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.307040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.307313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.315524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:05.318645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.320220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:05.337920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:05.339154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:05.340185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:05.345324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 350/700] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:05.345972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.346274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.347748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.348512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.349191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.369762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.372299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:05.398670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:05.399733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:05.400761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:05.400761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:05.400761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:05.408449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 351/701] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:05.408449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.514132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635edf1-d51d-4deb-bdb5-a4da41e146e4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.562125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.562794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.563089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.563345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.564050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:05.564886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.566910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.576200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:05.585881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 351/702] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:05.586841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.602781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18688b5c-fd41-458e-a343-83c4ff411778 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.651406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.652433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.653527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.654010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.654905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:05.655405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.657175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.673306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 352/703] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:05.674919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.716128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:05.757454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562d7b2-6ab4-43ea-afb0-3712e50a6670 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.815997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.817620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.818206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.818828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.820078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:05.820663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.821162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431b24f-a529-4529-88b3-84bffc476617 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:05.822651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.847254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 352/704] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:05.847254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:05.893192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:05.894105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:05.894537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:05.894952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:05.895980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:05.898491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:05.949195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:05.993992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01486b-540f-4eb3-a8b8-925eb9a177ff None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:06.028959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:06.029172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 353/705] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:16:05 2022] POST /identity/v3/users => generated 254 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:06.029800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.029936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.030070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:06.030249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.030863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=1468fb64bc6d4b1899be323c0116c8e0) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:06.031173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:06.032727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c121b61-9741-4544-a9c8-2da7b7d707d5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:06.052371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 353/706] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:05 2022] GET /identity/v3/role_assignments?user.id=1468fb64bc6d4b1899be323c0116c8e0&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:06.052371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea7f2-f1c1-4e40-8193-ef047bfd6dc8 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.052371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea7f2-f1c1-4e40-8193-ef047bfd6dc8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.052371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea7f2-f1c1-4e40-8193-ef047bfd6dc8 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.052371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 354/707] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:06.052371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98530856-9b21-4d28-89d4-42dd661a304a None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.052942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98530856-9b21-4d28-89d4-42dd661a304a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.052942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98530856-9b21-4d28-89d4-42dd661a304a None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.052942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 355/708] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:06.054757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bd54-2322-4a68-80ad-27e19283a287 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.056063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bd54-2322-4a68-80ad-27e19283a287 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.056063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0bd54-2322-4a68-80ad-27e19283a287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.094738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-cdf0bd54-2322-4a68-80ad-27e19283a287 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:06.096991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cdf0bd54-2322-4a68-80ad-27e19283a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:06.175195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:06.195088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68793707-7bdc-41f4-a208-7859903a9161 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:06.208993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 356/709] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:06.211110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.211402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.211724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.242687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:06.257131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.257131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.257131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.257410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:06.257410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:06.257544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:06.257544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:06.270133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:06.271701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:06.272989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 354/710] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:06.277658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:06.335383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:06.377599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 357/711] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:06.446500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4c720-7d87-4587-804c-904850022b87 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:06.492138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:06.493110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.493950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.494423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.495524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:06.496002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:06.499265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:06.522863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 355/712] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:06.530436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:06.566746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:06.581004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:06.716653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcede3-2aa0-46d1-8277-b0939b04c0f6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:06.744920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece70c50-e450-43b6-b0c2-157462bec6cd None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:06.760815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:06.761652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.762094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.762866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.763599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=f1427404350643b2892469b0da65b788) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:06.763912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:06.766266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:06.782208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 358/713] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:06 2022] GET /identity/v3/role_assignments?user.id=f1427404350643b2892469b0da65b788&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:06.783422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:06.792719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:06.794574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:06.794574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:06.794961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] PATH_INFO: `/v3/services/network` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:06.795608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:06.798340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:06.837515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:06.839475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:06.844062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 356/714] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:06 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:06.855326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:06.859106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:06.945981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ae12c-cdf0-4524-84a1-f4493b9e19e0 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:06.988827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:07.019735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:07.020676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:07.021162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:07.021612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:07.022562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:07.024988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7cdbb-9660-4d19-8321-54da35cf5b8f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:07.025303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:07.046133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:07.048913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:07.050411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 359/715] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 31 00:16:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:07.055686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:07.078637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:07.079895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:07.080912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:07.081294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:07.082084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] RBAC: Adding query filter params (name=network) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:07.082965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:07.087943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:07.100399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 357/716] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:07.104566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:07.268296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d01de-14dc-4d17-a444-090f2436ee56 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:07.304775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:07.324723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add73c52-a32a-4f70-8847-d14fcc66473f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:07.361560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:07.362130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:07.362253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:07.362466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:07.362782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:07.362922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:07.363090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:07.363373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:07.363471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:07.363761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:07.364008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] RBAC: Adding query filter params (type=network) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:07.364316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:07.365250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:07.365731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b2179-37c8-466c-b653-d7285ce444b3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:07.367500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:07.370729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 360/717] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 00:16:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:07.378635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 358/718] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:07.380918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:07.537486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43d050-3465-4db5-8e02-aec40dc6f82b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:07.584499 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:07.585059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:07.585336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:07.585604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:07.586259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:07.586561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:07.587976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7a41-5432-47f5-936b-240f573eede4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:07.595186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 361/719] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 00:16:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:09.006601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899821b2-4ede-44dc-8562-5adc5c826d56 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:09.007719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899821b2-4ede-44dc-8562-5adc5c826d56 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:09.008249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899821b2-4ede-44dc-8562-5adc5c826d56 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:09.010104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 359/720] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:09.018522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a520c-9f95-4767-a6d7-9e4ba0429bc4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:09.019023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a520c-9f95-4767-a6d7-9e4ba0429bc4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:09.019423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a520c-9f95-4767-a6d7-9e4ba0429bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:09.047202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-857a520c-9f95-4767-a6d7-9e4ba0429bc4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:09.049144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-857a520c-9f95-4767-a6d7-9e4ba0429bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:09.093622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:09.154484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 362/721] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:09.242264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b463-b8b3-457b-a154-34ab2059c584 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:09.242597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b463-b8b3-457b-a154-34ab2059c584 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:09.242886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b463-b8b3-457b-a154-34ab2059c584 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:09.244673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 360/722] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:09.251932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:09.252316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:09.252726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:09.284022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:09.285755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:09.392483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 363/723] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:09.719838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b463-b8b3-457b-a154-34ab2059c584 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:09.804254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:09.904305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b463-b8b3-457b-a154-34ab2059c584 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:09.978843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99392e2-c6ac-4895-a9e9-d6073b3b9e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:09.979767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99392e2-c6ac-4895-a9e9-d6073b3b9e7d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:09.980271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99392e2-c6ac-4895-a9e9-d6073b3b9e7d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:09.980841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99392e2-c6ac-4895-a9e9-d6073b3b9e7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:09.981873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:09.987021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d30a85-5b0c-4dc9-b40d-2781106a8845 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:09.997841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99392e2-c6ac-4895-a9e9-d6073b3b9e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e99392e2-c6ac-4895-a9e9-d6073b3b9e7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:10.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:10.005143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:10.006066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 361/724] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:16:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:10.008270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cee840-2825-4a21-bd66-939743d1dbf2 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.008270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cee840-2825-4a21-bd66-939743d1dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.008270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cee840-2825-4a21-bd66-939743d1dbf2 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.012308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 362/725] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:10.012308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cee840-2825-4a21-bd66-939743d1dbf2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.042236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6562c-42e3-43af-ac49-7f1b218bcb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.043724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6562c-42e3-43af-ac49-7f1b218bcb8c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.044445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6562c-42e3-43af-ac49-7f1b218bcb8c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.044867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6562c-42e3-43af-ac49-7f1b218bcb8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.057239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6562c-42e3-43af-ac49-7f1b218bcb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-6ec6562c-42e3-43af-ac49-7f1b218bcb8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:10.063734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:10.065878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:10.067750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:10.072959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 364/726] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:10.075152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.075544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.075913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.101669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:10.102911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:10.118474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:10.184889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cee840-2825-4a21-bd66-939743d1dbf2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:10.195438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 365/727] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:10.197142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.238912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.239733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.240219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.240808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.241954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:10.242388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:10.244645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:10.255209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 363/728] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:10.265513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.315476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:10.329421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c352a7-136e-4382-b771-79fa5f8a59c9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:10.377880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.379206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.379697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.380176 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.381080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:10.381592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:10.383496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175235c9-b127-414e-bcd8-74e9aeaebc1e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:10.407061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 366/729] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:10.407061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d18dc-c93c-40be-8e33-b97969d9a092 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.407061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d18dc-c93c-40be-8e33-b97969d9a092 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.407061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d18dc-c93c-40be-8e33-b97969d9a092 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.409181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 367/730] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:10.411822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d18dc-c93c-40be-8e33-b97969d9a092 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.470575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c7fe8-d8a0-44dc-aca5-d5c48b722ec4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:10.510109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:10.535534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.536532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.536982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.537456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.556138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:10.561240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-94d137d5-5eed-46ab-b9dc-3bd52a6717f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:10.581515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:10.583041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:10.584475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:10.584475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:10.584475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:10.584475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 364/731] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:10.584475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f18178-a959-4abb-a4d6-6832027c7031 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.584887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f18178-a959-4abb-a4d6-6832027c7031 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.584887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f18178-a959-4abb-a4d6-6832027c7031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.605195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d18dc-c93c-40be-8e33-b97969d9a092 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:10.606871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f4f18178-a959-4abb-a4d6-6832027c7031 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:10.608496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f4f18178-a959-4abb-a4d6-6832027c7031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:10.658344 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.659295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.659746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.660174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.675614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:10.677124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:10.693918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:10.695052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:10.696700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:10.696700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:10.696700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:10.696944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 365/732] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:10.696944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd410d6-83e0-4f7a-938c-b91f0cc9cc2d None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.696944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd410d6-83e0-4f7a-938c-b91f0cc9cc2d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.696944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd410d6-83e0-4f7a-938c-b91f0cc9cc2d None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.699819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 366/733] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:10.699819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd410d6-83e0-4f7a-938c-b91f0cc9cc2d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.700101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 368/734] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:10.706206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.711221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:10.860196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd410d6-83e0-4f7a-938c-b91f0cc9cc2d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:10.862390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:10.874204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d942bf9-dfe8-4fc5-a279-865408afda96 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:10.917746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.918556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.919082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.919364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.920040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:10.920812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:10.921888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:10.944281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 367/735] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:10.944520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138613af-bcf3-476f-9791-3368d9be4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:10.944666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:10.944802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138613af-bcf3-476f-9791-3368d9be4b83 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.944802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138613af-bcf3-476f-9791-3368d9be4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.944802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138613af-bcf3-476f-9791-3368d9be4b83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.962415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138613af-bcf3-476f-9791-3368d9be4b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-138613af-bcf3-476f-9791-3368d9be4b83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:10.969422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:10.971408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:10.973262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:10.976449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 369/736] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:10.977727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:10.978014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:10.978292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:10.998742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:11.000181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:11.067845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:11.078199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 370/737] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:16:11.079039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.086868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5e68c-9d4c-4d88-83cb-dade0a968bb0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.139082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.139703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.140137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.140514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.141239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:11.141605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.143473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.165160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 368/738] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:11.165160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.203298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:11.237833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685ecd-b85e-4279-a2a0-b8e8c7e000a6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.285409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.286861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.287589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.288292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.298328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae522bce-7011-4d35-8408-11cda3c9e626 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.301837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.304315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:11.319671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:11.321930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 371/739] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:16:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:11.323554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.350987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.351867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.352271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.352690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.353741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:11.354257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.357235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.371865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 369/740] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:11.373214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.402991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:11.458962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82b88-bfa6-4bc0-a496-71ffc830bb00 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.492936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:11.495446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.496171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.496776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.497369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.498625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.501125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.510191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156423a-e9a8-419f-af14-2242cc884e5e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.557238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.557905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.558182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.558455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.568261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.569753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:11.582678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:11.584841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 370/741] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:11.586918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.596999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 372/742] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:11 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:11.598095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.761132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98466dfa-bdf3-4404-84e7-d1e3ce15bb36 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.775729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c495ea-6930-410f-8f81-25a3f5670055 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:11.796640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:11.817155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.818013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.818441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.819604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.820130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:11.820568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.822512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.835199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:11.835199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:11.835673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:11.836114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:11.837270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:11.839653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:11.849743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 371/743] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 00:16:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:11.851300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:11.856944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:11.919028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 373/744] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:11 2022] POST /identity/v3/services => generated 251 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:11.919898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:12.024615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3a68f-33b5-42ec-8f56-0bacdda1b257 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:12.093404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1e3d1-6f43-44d3-bc3e-381d3adb94a8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:12.108824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:12.109904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:12.110387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:12.110867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:12.131782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:12.134390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:12.157271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:12.158252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:12.158705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:12.159157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:12.160536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:12.161673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:12.161792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:12.162695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:12.163581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:12.163581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:12.163581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:12.163581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 372/745] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:12.163824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:12.169218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:12.200942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 374/746] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 00:16:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:12.206634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:12.265378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:12.279201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:12.368702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5464f-efd4-438d-b015-f396a355d32d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:12.382075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd70800-55cb-480e-bb60-04f28d51302a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:12.438620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:12.440380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:12.440927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:12.441628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:12.450020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:12.450654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:12.451062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:12.451399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:12.452250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:12.452661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:12.454090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:12.476167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 375/747] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:12.477374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:12.579449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:12.583672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:12.634168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:12.662027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a49c-1260-4777-8f42-0f3a810ff39d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:12.669463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:12.718399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:12.719347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:12.719780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:12.720241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:12.753095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 373/748] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:12.754512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:12.802210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:12.805528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:12.893366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ca34-d766-4b29-9e7b-14aba6f7ab77 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:12.912281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:12.937603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 376/749] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:12.938690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:12.940281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:12.941677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:12.942590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:12.943283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:12.956575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:12.958285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:12.975690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:12.976918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:12.978181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:12.986376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 374/750] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:16:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:12.986376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.004684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:13.102151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f9218-31ff-43ea-b5c4-ba03794e7adc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.157139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.158388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.158931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.159639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.174905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e87e4-9992-4ca6-9939-3d6e46adc3b5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.176459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.180652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:13.211629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:13.213364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:13.216025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:13.216025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:13.216025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:13.220091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 377/751] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:13.221240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.239798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.241129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.241778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.242380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.263806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.266588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.275274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:13.286745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:13.288541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:13.289896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:13.289896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:13.289896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:13.289896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 375/752] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:13.289896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.395074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7c536-23d5-4c12-a249-175b66579b44 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.401230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:13.435727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.436251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.436553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.436823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.437558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:13.437863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.439324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.464370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 378/753] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:16:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:13.464370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.509918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0b93b-3225-45d8-a9e4-016d971fa21f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.573513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:13.586520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.587856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.588602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.588771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76bd7a-3bcb-44af-b347-7ee51ab109ea None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.589559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.591775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:13.592656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.595583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.624924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 376/754] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:13.626993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.640948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.641565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.641867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.642140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.642790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:13.643095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.645252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.658596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 379/755] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:13.659472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.779870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67ba8f-2d40-4ef8-b393-ddf295051fa2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.821448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.822092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.822374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.822648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.831329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.832865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:13.846305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:13.848079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 380/756] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:13.850002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:13.852381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8ca1d-7914-4bf6-ad38-e0c4f2a1b540 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.870104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:13.904468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:13.929605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:13.931171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:13.932030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:13.933302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:13.957900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:13.962352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:13.972626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78967d60-1f3c-45cb-ba06-80696020e34e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:13.995728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:13.998088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:13.999798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:13.999798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:13.999798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:14.005248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 377/757] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:14.013695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.018561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96371113-39df-4473-adfa-81d203532f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.019413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96371113-39df-4473-adfa-81d203532f35 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.019773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96371113-39df-4473-adfa-81d203532f35 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.020153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96371113-39df-4473-adfa-81d203532f35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.032228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96371113-39df-4473-adfa-81d203532f35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.035291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96371113-39df-4473-adfa-81d203532f35 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-96371113-39df-4473-adfa-81d203532f35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:14.050771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:14.052520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 381/758] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:14.054290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96371113-39df-4473-adfa-81d203532f35 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.132089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:14.167059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96371113-39df-4473-adfa-81d203532f35 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.210742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.211356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.211626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.211955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.212584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:14.212882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.214281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.233004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 382/759] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:14.233004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.254186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324e0a-d4fc-4211-b8fe-abfb9f30303c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.327905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.329475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.330255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.331142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.332901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:14.333781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.337176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.363611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 378/760] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:14.365631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.387446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11284a33-ff58-4b0b-b7b5-76003d6b247c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.424826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:14.437984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:14.445655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.446606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.446987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.447367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.448681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:14.449691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.452185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.468852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 383/761] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:14.470241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.519750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4821ce-025d-431b-8c0f-153f66e861a6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.560135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.561365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.561673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.562126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.572890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.574813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-eab99e94-65eb-4817-8bb3-71683be721a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:14.592227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:14.593249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 379/762] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:14.594581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1108ec-fe92-44be-8879-ef22b4d5b2e5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.595615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1108ec-fe92-44be-8879-ef22b4d5b2e5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.595615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1108ec-fe92-44be-8879-ef22b4d5b2e5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.600294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 380/763] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:14.601590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29585592-a092-4aba-85a6-bae828597800 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.601876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29585592-a092-4aba-85a6-bae828597800 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.602146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29585592-a092-4aba-85a6-bae828597800 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.603443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 381/764] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:14.604160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29585592-a092-4aba-85a6-bae828597800 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.606374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6dde51-90c8-451e-831f-ddc407cf9a58 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.656890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.657857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.658221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.658560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.659366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=589534f84a6143a2a194ccbd1c84373f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:14.659742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.661310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.667700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 384/765] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/role_assignments?user.id=589534f84a6143a2a194ccbd1c84373f&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:14.669147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:14.765783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:14.785395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29585592-a092-4aba-85a6-bae828597800 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.804307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:14.850388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.851177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.851726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.852198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.853382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=1468fb64bc6d4b1899be323c0116c8e0) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:14.853692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.855158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8dd2a-2e69-46db-b858-72b483fa7ae5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.863067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 382/766] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/role_assignments?user.id=1468fb64bc6d4b1899be323c0116c8e0&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:14.864414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.864694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.864958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.892777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3fa53-f283-4144-aab6-bdbb2da6ea16 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:14.895476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:14.897989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:14.953423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:14.954060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.954329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.954626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:14.955318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:14.955660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:14.957297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de400a-baba-4b37-bcf1-a60ee787d6e6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:14.985233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 385/767] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:14.985233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:14.985233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:14.985233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:15.016022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:15.018476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:15.029639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 383/768] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:15.030856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:15.114207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 386/769] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:15.207888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:15.223379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6873de-6e40-4664-a829-b3f8ed7433b6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:15.305741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:15.306549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:15.306721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:15.306982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/f1427404350643b2892469b0da65b788/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:15.310078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=f1427404350643b2892469b0da65b788, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:15.311806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:15.349497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:15.421592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:15.442851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 384/770] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:16:15 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/f1427404350643b2892469b0da65b788/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:16:15.512316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2695485-2e8b-4ca2-8ef6-ff98128fb52f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:15.558900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:15.564354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:15.565215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:15.565708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] PATH_INFO: `/v3/services/network` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:15.566852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:15.568642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:15.584355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:15.585453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:15.586320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:15.592084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 387/771] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:15 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:15.597829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:15.646567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:15.874393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e931-7cc9-4f96-a71e-7233ed54badc None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:15.900739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc00cb-e002-41c8-871a-864bab738ec1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:15.917084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:15.917773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:15.918101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:15.918420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:15.919094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] RBAC: Adding query filter params (name=network) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:15.919404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:15.920869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:15.927386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 385/772] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:15.932876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:15.936092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:15.963313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:15.964122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:15.964443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:15.965493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:15.966722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:15.968058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:15.971628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:15.981916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:15.986175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:15.987215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 388/773] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 31 00:16:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:15.991248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:16.179972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672e426-5c0c-41ae-959b-c5f532eed8eb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:16.239987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddca3f-9eac-47ad-8519-db271c0ea52f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:16.285145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:16.285788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:16.286068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:16.286341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:16.286992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] RBAC: Adding query filter params (type=network) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:16.287298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:16.288717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:16.290844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:16.291469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:16.291742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:16.292009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:16.292830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:16.293656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:16.295096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:16.302918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 386/774] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:16.304927 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 389/775] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 00:16:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:16.311464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:16.313619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:16.452822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:16.529698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:16.569734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f2b98-aa73-43c8-a177-5869de18ee7a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:16.586409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f74c22-c9e6-41fb-85a9-bbf304207732 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:16.626821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:16.627472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:16.627793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:16.628075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:16.628771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:16.629307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:16.630653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:16.636313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 390/776] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 00:16:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:16.642349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:16.654224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:16.655244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:16.655739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:16.656239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:16.657187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] RBAC: Adding query filter params (interface=public, service_id=8622f90cad684557a46e2776c2deb231, region_id=RegionOne) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:16.657705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:16.659333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:16.676721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 387/777] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:16 2022] GET /identity/v3/endpoints?service_id=8622f90cad684557a46e2776c2deb231&interface=public®ion_id=RegionOne => generated 194 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:16.682099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:16.761954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50436ca1-6309-4183-ba15-2a60cd02276f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:16.764698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:16.802457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:16.803096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:16.803388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:16.803668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:16.804408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] RBAC: Adding query filter params (interface=public, service_id=5c45215d26ca4d83a7c0e0f4ebbbac96, region_id=RegionOne) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:16.804727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:16.806327 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:16.815654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 391/778] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:16 2022] GET /identity/v3/endpoints?service_id=5c45215d26ca4d83a7c0e0f4ebbbac96&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:16.822387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:17.001534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d341a16-fc4b-4db2-916c-2a19e8140806 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:17.001762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc124165-9255-4575-881d-d673db9a2776 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:17.074867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92066b19-d342-4451-b263-033fc9d2b43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:17.075873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066b19-d342-4451-b263-033fc9d2b43c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.076393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066b19-d342-4451-b263-033fc9d2b43c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.077622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066b19-d342-4451-b263-033fc9d2b43c None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.078762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92066b19-d342-4451-b263-033fc9d2b43c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:17.081509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92066b19-d342-4451-b263-033fc9d2b43c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:17.092362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 388/779] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:16 2022] GET /identity/v3/services => generated 772 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:17.112253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dbed5-9c9b-4247-aa4d-b11b7c2b5333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:17.112959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dbed5-9c9b-4247-aa4d-b11b7c2b5333 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.113237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dbed5-9c9b-4247-aa4d-b11b7c2b5333 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.113512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dbed5-9c9b-4247-aa4d-b11b7c2b5333 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.114135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dbed5-9c9b-4247-aa4d-b11b7c2b5333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:17.121885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dbed5-9c9b-4247-aa4d-b11b7c2b5333 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:17.122148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499380f-657c-4503-90d6-c84340fb8757 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.122532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499380f-657c-4503-90d6-c84340fb8757 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.122911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499380f-657c-4503-90d6-c84340fb8757 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.125716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 389/780] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:17.131861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf44d7-cefa-487b-8ae6-04423a98f855 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.132173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf44d7-cefa-487b-8ae6-04423a98f855 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.132437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf44d7-cefa-487b-8ae6-04423a98f855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.132993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 392/781] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:16 2022] GET /identity/v3/services => generated 772 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:17.149046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:17.165768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f2cf44d7-cefa-487b-8ae6-04423a98f855 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:17.167184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f2cf44d7-cefa-487b-8ae6-04423a98f855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:17.303012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 390/782] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:17.434243 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9188f9-e1e6-413b-961e-eb5263e1d35f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.435063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9188f9-e1e6-413b-961e-eb5263e1d35f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.435359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9188f9-e1e6-413b-961e-eb5263e1d35f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.437606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 393/783] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:17.444989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.445522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.445522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.466947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:17.468191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:17.585559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 391/784] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:17.713026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9188f9-e1e6-413b-961e-eb5263e1d35f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:17.760346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:17.868402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9188f9-e1e6-413b-961e-eb5263e1d35f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:17.929484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:17.930465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:17.930906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:17.931335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:17.962039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:17.966655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:17.971875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:17.974149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:17.976108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 394/785] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:16:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:17.985299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:18.124752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:18.162701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:18.178652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ce04-f196-42f7-9bd5-d2a9f018683d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:18.186333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64525ca0-6e12-4a26-aeea-3cb504b9e2f9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:18.219815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:18.220378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.220904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.221328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.222500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:18.222662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:18.224135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f2a0b-3b24-4bf0-b49d-1f416c3029c7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:18.231893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046cf86-1a44-4794-8d2e-fe8d9ee295c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:18.232865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046cf86-1a44-4794-8d2e-fe8d9ee295c7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.233289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046cf86-1a44-4794-8d2e-fe8d9ee295c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.233775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046cf86-1a44-4794-8d2e-fe8d9ee295c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.240399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 395/786] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:16:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:18.240399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1514e83-f9c9-447f-b8b0-3a465db1c17b None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.240399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1514e83-f9c9-447f-b8b0-3a465db1c17b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.240399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1514e83-f9c9-447f-b8b0-3a465db1c17b None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.243823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 396/787] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:18.244828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1514e83-f9c9-447f-b8b0-3a465db1c17b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:18.254056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046cf86-1a44-4794-8d2e-fe8d9ee295c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-7046cf86-1a44-4794-8d2e-fe8d9ee295c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:18.259554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:18.260611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:18.261888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:18.270472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 392/788] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:18.272435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.272972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.273445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.295736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:18.297915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:18.393295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 393/789] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:18.394726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:18.410605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1514e83-f9c9-447f-b8b0-3a465db1c17b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:18.463792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:18.496136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:18.496136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.496136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.496136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.517432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:18.519905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:18.553570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:18.554607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:18.555477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:18.555477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:18.555477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:18.555477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 397/790] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:18.563436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:18.568539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:18.582024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13c1f6-3c6b-421c-a266-251309a273dc None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:18.647326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:18.648929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.649515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.650208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.651177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:18.651714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:18.654269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:18.666027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 394/791] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:18.672318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:18.790722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14224fe-cd60-4508-93ac-09ee088dec31 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:18.813701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad494-b0cd-4339-845e-14fdc1e77a65 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:18.865159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:18.866177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.866647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.867118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.868305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:18.868891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:18.871924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:18.896730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 398/792] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:18.896730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:18.950699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:18.971376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:18.973046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:18.973939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:18.974443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:18.974963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:18.987422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:18.989570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:19.009268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:19.010295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:19.011207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:19.011207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:19.011207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:19.016738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 395/793] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:19.017953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.018340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e148bb-15e0-4bf2-b430-aa228e3fe398 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.064656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.065601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.066346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.066790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.079637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:19.084989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:19.085998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:19.086845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:19.090427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 399/794] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:19.091530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.171004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08db68f-e6ad-4e66-abd5-4a4d645765b4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.214110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.216105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.216666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.217164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.227832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.229692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:19.243286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:19.244313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:19.246121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:19.252605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00a5b-ba95-4386-898e-3bab28988c09 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.252965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 396/795] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:19.254149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.290823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:19.292011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:19.309383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.311112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.311769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.312611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.315204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:19.316710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.320751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.348208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 400/796] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:19.349550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.412619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c3976-d548-4aad-baed-d5276cbcc645 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.456172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.457279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.457773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.458225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.459420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:19.460162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.462137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.473501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 397/797] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:19.475296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.485900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e9b7f-fc10-4a05-816d-fabffb8e68f5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.536674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.537439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.537778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.538118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.538945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:19.539310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.541216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.567702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 401/798] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:19.570547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.600184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:19.612823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2af3-84fd-491e-b110-c71dd4361f92 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.632447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:19.654852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.655506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.655787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.656056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] PATH_INFO: `/v3/users/swift` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.663975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.665467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:19.688070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:19.689322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 398/799] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:19.691193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.735899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7718452-8d29-409c-beea-96ec00a63d2f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.776829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.778838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.779290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.779680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.794723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.796216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b0590ffb-89ab-40a3-a452-b21a14f448e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:19.814073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:19.815566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:19.816766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:19.816766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:19.816766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:19.816766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 402/800] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:19.817443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab422-45e8-4baf-9d48-53ca8dbf9487 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.817443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab422-45e8-4baf-9d48-53ca8dbf9487 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.817619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab422-45e8-4baf-9d48-53ca8dbf9487 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.818933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 403/801] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:19.819434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ab422-45e8-4baf-9d48-53ca8dbf9487 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.842823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76255f7e-9505-4aab-993b-7344d179b03d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:19.903108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:19.904597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:19.905270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:19.905710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:19.924530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:19.926542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:19.951425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:19.953127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:19.956061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:19.956061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:19.956061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:19.956061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 399/802] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:19.956061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:19.958506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.003275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.012259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ab422-45e8-4baf-9d48-53ca8dbf9487 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.071803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.072754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.073036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.073928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.074927 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:20.075408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.077205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26bf97-d10f-41bc-bf9c-d12864a20061 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.100887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 404/803] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:20.102245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079834a6-e762-428b-866d-2153ff727701 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.102533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079834a6-e762-428b-866d-2153ff727701 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.102804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079834a6-e762-428b-866d-2153ff727701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.119611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799c80-a02c-419f-8112-d386aa70416d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.129759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-079834a6-e762-428b-866d-2153ff727701 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:20.130999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-079834a6-e762-428b-866d-2153ff727701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:20.162592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.163210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.163482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.163747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.164419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:20.164715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.166095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.179403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 400/804] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:20.180266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.205119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 405/805] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:20.205998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079834a6-e762-428b-866d-2153ff727701 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.265323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.305297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.312627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0a36c-a17f-4233-b841-7e5de9433cb6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.326425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079834a6-e762-428b-866d-2153ff727701 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.351797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.352598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.352974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.353710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.354650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:20.355471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.357399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26092a9c-e436-428d-a80f-19a4edb5485b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.368955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.369778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.370026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.370293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.382008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 401/806] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:20.383754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97210081-b907-4933-accd-bfbf2ccbac79 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.384282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97210081-b907-4933-accd-bfbf2ccbac79 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.384580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97210081-b907-4933-accd-bfbf2ccbac79 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.388370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 402/807] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:20.388370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97210081-b907-4933-accd-bfbf2ccbac79 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.394897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.398558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:20.416552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:20.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:20.419103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:20.419103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:20.419103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:20.423748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 406/808] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:20.424902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.511923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97210081-b907-4933-accd-bfbf2ccbac79 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.535996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.541521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f353f0c-5ade-422e-acb8-c81fe8bf8850 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.550047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.550684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.550954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.551218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.559311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.560830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-09a7b91a-69e7-4270-b537-a53cbe2d6171 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:20.575271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:20.576292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:20.577375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 403/809] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:20.577967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.577967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.578105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.590108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.591193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.591659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.592107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.595314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.604120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:20.615462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.615679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:20.615760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:20.629226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:20.637152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 407/810] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:20.638100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.687612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 404/811] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:20.688574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.755193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6454b-1fef-4505-ab64-bc4599d5517c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.801727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.802340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.802642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.802911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.803548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:20.803852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.805253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc70865-73a4-4caa-a3d8-5fbfd8e9130f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:20.805375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.818971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 408/812] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:20.819813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.826143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.854079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:20.854829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:20.855110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:20.855556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:20.856272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:20.857656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:20.866680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:20.867787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:20.868706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 405/813] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:20.869610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:20.901311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:20.970963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670bced-c5f2-4327-b392-a403110b2988 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.006337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195234d1-3c66-4ef1-b8eb-c45d067f872b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.009676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.010278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.010553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.010849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.011588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:21.011892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.013362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.031128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 409/814] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:21.032035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.044545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.045230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.045503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.045765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.046464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:21.046765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.048197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.061908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 406/815] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:21.061908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.103729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.139674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc018-9b9f-4294-9ddc-b366773f5c3d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.181851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c5f9-251a-4026-8d46-ef63088f2b8c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.223374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.224003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.224283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.224585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.225233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:21.225533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.227016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.233101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 407/816] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:21.233971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.274223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.299345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.299972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.300250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.300542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.307489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.309659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:21.322279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:21.323415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 410/817] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:21.325098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.348986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.352062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea83e-222d-4c63-9e49-77f0dc2ab07d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.388232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.388898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.389169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.389436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] PATH_INFO: `/v3/services/network` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.390064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.391369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:21.400472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:21.401704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:21.402561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 408/818] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:21.403349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.429042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ab7d0-236c-4c2a-bbd5-e5db4ee85482 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.463880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.465114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.465399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.465665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.514058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.516974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090085e-e31f-4acd-b918-da00b9ae5b37 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.543116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.544802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.553324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.553992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.554264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.554532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/589534f84a6143a2a194ccbd1c84373f/roles/deb6dc5f801e411882108de755caff96` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.570683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=589534f84a6143a2a194ccbd1c84373f, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.572212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.649503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 411/819] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:21.650294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.663675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 409/820] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:16:21 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/589534f84a6143a2a194ccbd1c84373f/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:16:21.664562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.705409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.731692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.767274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a0b4d-b8b9-40ef-bdb1-6460fbefde9e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.810135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a2ef5-7179-4e1a-a85f-455927c6b98c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.811000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.811624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.811900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.812167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.812865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:21.813166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.814545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.822155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 412/821] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:21.823110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.869513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.870588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.871570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.872060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.873348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:21.873959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.876608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.902142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 410/822] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:21.902142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:21.944063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345accc-a1a1-4094-9710-e1ebe8b08f8a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:21.982054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:21.982678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:21.982954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:21.983224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:21.983893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] RBAC: Adding query filter params (name=network) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:21.984206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:21.985613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:21.987650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.996686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 413/823] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:21.996864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:21.996946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.019359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a728f1-3dcd-4643-b5d4-201360b1a0f8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.054966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.055605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.055879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.056146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.072469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.074152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:22.091767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:22.097567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:22.098740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:22.098740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:22.098740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:22.098740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 411/824] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:22.098740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.115741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc275-782b-474b-a2ac-6158f1374572 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.154118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.154963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.155399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.155667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.156358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.157740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.212221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0452d1-b234-457b-a1cf-976c8d5b2ed7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.239084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:22.248114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.248800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.249071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.249335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/1468fb64bc6d4b1899be323c0116c8e0/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.251227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=1468fb64bc6d4b1899be323c0116c8e0, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.252671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.288727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 414/825] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:22.289760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.357182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:22.365978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 412/826] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:16:22 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/1468fb64bc6d4b1899be323c0116c8e0/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:16:22.365978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.485420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84962710-5c0b-4067-b03b-a576189ee398 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.527570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa8f7c-5ee5-413b-9b93-52b0459e5114 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.532652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.533565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.533996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.534449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.535259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:22.536402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.538309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.546710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 415/827] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:22.552354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.579940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.580990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.581271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.581537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.582202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:22.582499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.583884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.604350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 413/828] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:22.609792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.662614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:22.688585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dc45c-2f00-42ca-ad23-a68216cd69e4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.707230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:22.722141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f6bff-18f3-495a-ad69-1c2144f8d724 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.727710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.728537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.728907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.729286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.730292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.731926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.760261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:22.761049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:22.762075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:22.762444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:22.770406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:22.772069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:22.785551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:22.790786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:22.791902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 414/829] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:22.793882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:22.843407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 416/830] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:16:22.963284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b15fc4-e4b1-42e9-8065-14d3dd900a4f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:22.978138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:23.030024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:23.030564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:23.030842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:23.031110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:23.031812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:23.032115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:23.033779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed6842-2dea-4bb9-9373-33fb3a97a2d8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:23.052822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 415/831] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:23.060142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:23.189665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ad713-809f-46a7-a301-395736498bb7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:23.226672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:23.227647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:23.228143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:23.228817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:23.229947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=f1427404350643b2892469b0da65b788) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:23.231011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:23.232750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fd7df-f749-40d8-ab22-c1abee08e213 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:23.240020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 417/832] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:23 2022] GET /identity/v3/role_assignments?user.id=f1427404350643b2892469b0da65b788&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:23.793217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67177493-6eba-496c-b19b-8f321a06f685 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:23.793782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67177493-6eba-496c-b19b-8f321a06f685 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:23.793782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67177493-6eba-496c-b19b-8f321a06f685 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:23.795410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 416/833] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:23.800917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:23.801201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:23.801465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:23.821264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:23.822503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:23.853564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:23.898328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 418/834] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:24.172924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67177493-6eba-496c-b19b-8f321a06f685 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:24.283261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67177493-6eba-496c-b19b-8f321a06f685 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:24.320113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:24.320791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.321065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.321333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.334605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:24.339598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:24.340544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:24.341379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 417/835] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:16:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:24.347483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:24.466712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6416c9-93e3-493c-ba4e-01f3daec3340 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:24.504656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:24.505274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.505556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.505831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.506535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:24.506837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:24.508234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:24.518475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 419/836] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:16:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:24.523738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:24.551472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:24.637600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfb95f-a116-49fc-9862-f9ab088d6295 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:24.678341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:24.679018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.679291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.679610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.691384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:24.693287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-c9b8956b-f6b8-413f-a138-02ad2a9ce7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:24.710249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:24.711257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:24.712139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:24.712139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:24.712139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:24.714869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 418/837] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:24.720630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:24.752450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a83714-743f-48d9-82f0-70d59f9aa4c8 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.752778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a83714-743f-48d9-82f0-70d59f9aa4c8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.753006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a83714-743f-48d9-82f0-70d59f9aa4c8 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.753254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:24.754350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 419/838] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:24.761411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf391e0-18a1-40b5-89d9-b5d009faf813 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.761791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf391e0-18a1-40b5-89d9-b5d009faf813 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.772874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf391e0-18a1-40b5-89d9-b5d009faf813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.786173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-baf391e0-18a1-40b5-89d9-b5d009faf813 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:24.787425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-baf391e0-18a1-40b5-89d9-b5d009faf813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:24.845526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002eed4-b908-4e86-87d1-37bb6c2b41d7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:24.864853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 420/839] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:24.865940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5f26a-5fee-4496-aa8d-85410fb96e08 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.866228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5f26a-5fee-4496-aa8d-85410fb96e08 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.866489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5f26a-5fee-4496-aa8d-85410fb96e08 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.872113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 421/840] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:24.873678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.873953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.874215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.884187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:24.893719 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:24.895157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:24.895745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:24.896244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:24.896504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:24.897322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:24.897690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:24.897966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:24.900972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:24.923565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 420/841] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:24.925303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:24.996617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 422/842] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:25.045268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.057366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5afedc-4a0c-4f32-938c-5c89d4d8c818 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.097268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.097958 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.098236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.098500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.106952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.108531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:25.121763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:25.122861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:25.123802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 421/843] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:16:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:25.129026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.141369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.245086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd98fcc-50a0-443e-aa53-e52991a4b080 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.258802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b95b4e-bd2c-4f54-9c47-8629663f99cc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.284340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.284957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.285228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.285492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.286320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:25.286624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.288147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.290065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.306429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.308023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.308836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.309633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.311013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.313039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.314019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 423/844] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:25.315321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:25.324377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:25.325446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:25.326282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:25.327536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 422/845] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:25.327536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.344926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.432397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e8d52-4a8c-4b8e-9b7f-b2c9c4ee659f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.445077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd415b-2671-4d99-a144-2a9a013131fe None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.469560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.470176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.470592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.470797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.481714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.482362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.482624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.482899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.493652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:25.498493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:25.499438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:25.500254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 423/846] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:25.501499 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.539697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.549680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.551131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.575291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.608649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e58e32-aa47-4a8f-8f02-363cf7a30a4c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.646107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.646716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.646993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.647264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.647897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:25.648194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.649609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd654eaa-350c-4db0-acff-8f07a572b00a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.655155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 424/847] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:25.656382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad3dd-2b16-4807-958a-1b520f18e4b6 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.656726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad3dd-2b16-4807-958a-1b520f18e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.656930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad3dd-2b16-4807-958a-1b520f18e4b6 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.660396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 425/848] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:25.660396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad3dd-2b16-4807-958a-1b520f18e4b6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.664024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 424/849] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:25.665067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.772750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad3dd-2b16-4807-958a-1b520f18e4b6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.777416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.786254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361970d-6f04-4254-9880-187affdb25bc None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:25.812423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.812964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.813238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.813507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.814158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:25.814454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.816178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a62d55-d1ab-455b-a29e-bdb0f588fadd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.818314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:25.821795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:25.822397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.822675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.822977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.823804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:25.824350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:25.825234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 426/850] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:25.826246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:25.826437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:25.826671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:25.826938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:25.832191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 425/851] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:25.833163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.848714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:25.849964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:25.945934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 427/852] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:25.947029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:25.986398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515959c-067a-49f3-a58a-9752e93dbe2d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.036702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.037658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.038091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.038512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.052084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.054647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:26.077550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:26.078874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:26.079979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:26.079979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:26.079979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:26.082736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 426/853] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:26.090929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.108312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:26.114688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552237e-fdb4-425e-9363-e12893894f3d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.148839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:26.158354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.159133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.159473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.159810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.170954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.172659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:26.190485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:26.191810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:26.193193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:26.193193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:26.193193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:26.193193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 428/854] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:26.199385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.239252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab028ba7-0c46-493a-b87f-39b2373797d2 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.276770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.277470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.277758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.278032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.278678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:26.278981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.280398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.294687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 427/855] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:26.299843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.317272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724ae77-a61d-484c-a222-2aa7c3c51e66 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.366966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.366966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.366966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.366966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.367395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:26.367395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.368932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.384231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:26.384486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 429/856] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:26.385711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.425933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515060e-7de9-47d2-9205-be83df2d924b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.441989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:26.487477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.488116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.488419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.488732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.489320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.490682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:26.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:26.514150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:26.515018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 428/857] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:26.515812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.539752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f641cf-9414-4c85-9df1-017b44a45442 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.586170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.586822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.587208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.587385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.599925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.601712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:26.617632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:26.618587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:26.619469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:26.624016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 430/858] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:26.625467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.661469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306b275-61a2-4a7a-b867-ee602d21edc8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.700596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:26.707116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.707934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.708215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.709027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] PATH_INFO: `/v3/users/swift` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.720247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.722791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:26.739104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:26.740279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 429/859] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:26.747663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.748350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f6d2d-4ec6-48fd-bced-212811e7781d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.749287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:26.788549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.790262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.790262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.790484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.791077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:26.791568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.794694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.808279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 431/860] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:26.808279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:26.878471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5e400-9896-4375-9dd0-a629af81fcea None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.942544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:26.943477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:26.943903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:26.944344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:26.944776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba7d3c-900c-4a86-b925-983b8303430c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:26.945394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:26.945840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:26.947728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:26.968562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 430/861] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:26.970293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:27.010771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:27.012903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:27.012903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:27.013127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:27.014373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:27.014922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:27.017535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:27.042035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 432/862] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:27.045298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:27.069469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:27.084541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:27.143846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a861394-8593-42bb-80c6-bff986bc6606 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:27.199186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd42b-cea4-4ebd-8677-ec55b8a04099 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:27.213550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:27.215350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:27.215856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:27.216355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:27.217719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:27.218264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:27.220835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:27.230440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 431/863] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:27.231657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:27.254180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:27.254945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:27.255299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:27.255648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:27.256754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=589534f84a6143a2a194ccbd1c84373f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:27.257188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:27.258824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:27.272896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 433/864] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:27 2022] GET /identity/v3/role_assignments?user.id=589534f84a6143a2a194ccbd1c84373f&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:27.437002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06510e-6cf2-43af-a350-ab4d7552b9af None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:27.458485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:27.486947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:27.487513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:27.487791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:27.488066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:27.615424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:27.616935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:27.748422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 432/865] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:27.750527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:27.850269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:27.868081 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fdef6-f06b-4ca7-a003-adefeecc7107 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:27.906766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:27.907760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:27.908240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:27.908751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:27.922124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:27.923959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:27.939864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:27.941210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:27.942094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:27.942094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:27.942094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:27.943986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 434/866] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:27.950766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:27.970452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:28.070585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb966f-191a-4c1a-a5f0-2595635408dc None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:28.110833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:28.111555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.111836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.112104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.112774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:28.113068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:28.114429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28cf59-2d54-48be-b6b6-adc43d390cd1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:28.128276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 433/867] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:28.133784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:28.175227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6682832-7ec8-497e-ba06-e68747296fbe None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.175543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6682832-7ec8-497e-ba06-e68747296fbe None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.175769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6682832-7ec8-497e-ba06-e68747296fbe None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.177456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 434/868] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:28.183025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.183393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.183630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.203661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:28.205010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:28.257860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8e70-5bd5-45d4-98a0-b0b234199741 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:28.288739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 435/869] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:28.304041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:28.305082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.305619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.306085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.314351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:28.316659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:28.318829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:28.331102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:28.332049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:28.333049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:28.335178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 435/870] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:28.341960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:28.351959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:28.454366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f39957-9dfd-48c2-980d-b6a98b1e723e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:28.496949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:28.498443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.499192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.499967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.501612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:28.502441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:28.505484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:28.529526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 436/871] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:28.539831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:28.583753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:28.677489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:28.718133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5191a34-00cc-4685-a5ba-76239757eb6c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:28.725578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0d42-c808-4cb4-bf83-89b8ceb74af8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:28.766828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc28e5-e8ef-4c6f-80eb-8eab3d1c647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:28.767435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc28e5-e8ef-4c6f-80eb-8eab3d1c647b None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.767706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc28e5-e8ef-4c6f-80eb-8eab3d1c647b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.767968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc28e5-e8ef-4c6f-80eb-8eab3d1c647b None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.768722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc28e5-e8ef-4c6f-80eb-8eab3d1c647b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:28.770200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc28e5-e8ef-4c6f-80eb-8eab3d1c647b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:28.777289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:28.778210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:28.778664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:28.778918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:28.779729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=1468fb64bc6d4b1899be323c0116c8e0) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:28.780101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:28.781649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf1389-3ebc-4bbc-b739-f37042a5aea8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:28.787988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 436/872] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:28 2022] GET /identity/v3/role_assignments?user.id=1468fb64bc6d4b1899be323c0116c8e0&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:28.848267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 437/873] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:28 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:29.489321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb400-daee-4124-a1a2-e707a2c07223 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:29.490245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb400-daee-4124-a1a2-e707a2c07223 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:29.490754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeb400-daee-4124-a1a2-e707a2c07223 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:29.492676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 437/874] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:29.499521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027448-bba5-4063-924e-d49b2f983207 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:29.500462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027448-bba5-4063-924e-d49b2f983207 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:29.500972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027448-bba5-4063-924e-d49b2f983207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:29.520557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-07027448-bba5-4063-924e-d49b2f983207 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:29.522111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-07027448-bba5-4063-924e-d49b2f983207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:29.611373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 438/875] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:29.653936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d504-1145-4544-ab55-c77cb25438cc None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:29.654578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d504-1145-4544-ab55-c77cb25438cc None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:29.655077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d504-1145-4544-ab55-c77cb25438cc None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:29.656945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 438/876] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:29.662370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:29.662653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:29.662926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:29.683281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:29.684707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:29.742079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:29.766178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 439/877] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:29.903162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d504-1145-4544-ab55-c77cb25438cc None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:29.980053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:30.025859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d504-1145-4544-ab55-c77cb25438cc None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:30.066633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:30.067260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.067540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.067810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.068524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:30.070010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:30.134803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:30.188316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 439/878] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:29 2022] POST /identity/v3/services => generated 269 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:30.282011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e1be5-60c6-4536-9f76-4a4e7cb138fe None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:30.338684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:30.339564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.339839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.340108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.354222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:30.360616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:30.361821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:30.362988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 440/879] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 00:16:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:30.370754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:30.505885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b66ece-a5b1-4144-8cf9-92422a40a074 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:30.536831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:30.547327 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:30.548150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.548540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.548816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.549590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:30.550112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:30.551845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:30.563664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 440/880] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:16:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:30.571779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:30.618150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:30.687065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb2af8-b4fa-4b20-92a5-d4dd1a6b109f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:30.727544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:30.728179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.728472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.728759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.740563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:30.742019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-59082e05-38a5-4534-a5f6-7c43043b3eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:30.758271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:30.760558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:30.762664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:30.762664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:30.762664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:30.762664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 441/881] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:30.766782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:30.853753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fd38-a3f2-4961-a0ea-045d622ea701 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.855009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fd38-a3f2-4961-a0ea-045d622ea701 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.855664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fd38-a3f2-4961-a0ea-045d622ea701 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.857922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 442/882] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:30.863050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca1db-b0db-493c-923b-62f94f2c98ad None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.863456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca1db-b0db-493c-923b-62f94f2c98ad None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.864146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ca1db-b0db-493c-923b-62f94f2c98ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.884912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc345b-832e-4d51-8d9d-ad4debd9c6ff None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:30.889983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-df5ca1db-b0db-493c-923b-62f94f2c98ad None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:30.891311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-df5ca1db-b0db-493c-923b-62f94f2c98ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:30.928742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:30.929691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:30.930276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:30.930774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:30.931613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:30.932118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:30.934619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:30.956094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 441/883] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:30.963117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:30.998329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 443/884] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:31.018784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c431a70-e4f5-44c8-9e87-825e5f410d66 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.019068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c431a70-e4f5-44c8-9e87-825e5f410d66 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.019344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c431a70-e4f5-44c8-9e87-825e5f410d66 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.024955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 444/885] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:31.026438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.027148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.027424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.033106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.048754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:31.050170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:31.071428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.098889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4851f-31d6-4c49-b914-2bcc2fb4b8c5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.145771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.146675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.147130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.147604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.152386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 445/886] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:31.165634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.167194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:31.179131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:31.180099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.181130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 442/887] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:16:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:31.188101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.277521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.307255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215be63-efcb-4e6f-a103-dc132b2882ea None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.355674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.356965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.357424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.357724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.358580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:31.359093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.361500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:31.387302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 446/888] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:31.393908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.413196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.413596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c25a1-29d6-4fc6-95ed-d6027f35747f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.417359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.460601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.461287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.461614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.461880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.462527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.464047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:31.473572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:31.474609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:31.475456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.479817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 443/889] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:31.480795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.525495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb725fe-0618-4f4b-a810-3ad084211f29 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.578404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.579501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.580162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.580788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.624218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3b755-f897-49c7-b480-888cfed8c3be None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.670506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.671073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.671357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.671636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.672252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.673749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:31.674479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.676171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:31.684829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:31.686125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:31.687103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 444/890] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:31.688038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.692150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.726093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.803872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 447/891] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:31.803872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.815490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340faedb-f044-4bb9-b653-8ad69054b6e4 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.868563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.869304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.869636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.869914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.871002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:31.871308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.872985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:31.883607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 445/892] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:31.883607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:31.953907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2a770-2b03-4846-b54a-0e57e49657c1 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:31.964977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:31.994194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:31.994846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:31.995685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:31.996085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:31.996815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:31.997202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:31.998814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.004431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 448/893] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:32.005283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.012046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:32.018084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd289-e0b6-453c-97ae-db0952989536 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.056479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.057118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.057395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.058250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.070858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.072936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:32.091238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:32.092289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:32.098207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:32.098207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:32.098207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:32.098207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 446/894] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:32.098207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.129643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70622f99-e9ab-4a57-bd57-fad1cb27a66a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.172748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.173356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.173633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.173899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.174527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:32.174847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.176176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.188554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 449/895] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:32.189771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.233134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b6ad9-1d6f-40c6-9535-1ddf7da58e2c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.242787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:32.281654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.282700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.283629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.284477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.285352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:32.285866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.288094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.295649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 447/896] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:32.301461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.321241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:32.340962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cebb2-71e9-4608-8cdc-a1883291a366 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.402785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.404268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.405094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.405574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.406742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:32.407837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.410146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.427640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 450/897] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:32.431237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ff9d-fe6d-4065-a25a-f76312023b88 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.434165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.477784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.481651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.481651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.481651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.482620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] RBAC: Adding query filter params (interface=public, service_id=0a82f18b2547414aa767cd78c1875962, region_id=RegionOne) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:32.483188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.485575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.500768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 448/898] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/endpoints?service_id=0a82f18b2547414aa767cd78c1875962&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:32.505249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.582303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:32.591092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015397b-7d13-4d8d-93f2-8d6e3693fde9 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.618372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:32.628169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.628838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.629154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.629422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.639039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.641317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.644525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a30b-017d-4713-8664-cf82c876f273 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-486cd305-e374-44bb-81ed-7fd3249f60ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:32.654384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:32.655433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:32.656840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:32.660405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 451/899] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:32.661759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d06c5-609b-4ede-99f9-e24d823bffcb None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.662039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d06c5-609b-4ede-99f9-e24d823bffcb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.662311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d06c5-609b-4ede-99f9-e24d823bffcb None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.668798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 452/900] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:32.668798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d06c5-609b-4ede-99f9-e24d823bffcb None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:32.696919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e252-12ee-4f94-843c-4065c7f12a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.697590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e252-12ee-4f94-843c-4065c7f12a62 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.697870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e252-12ee-4f94-843c-4065c7f12a62 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.698731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e252-12ee-4f94-843c-4065c7f12a62 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.699595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e252-12ee-4f94-843c-4065c7f12a62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.701018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e252-12ee-4f94-843c-4065c7f12a62 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.707634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 449/901] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/services => generated 1268 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:16:32.717950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.719207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.719660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.774611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:32.774611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:32.829825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d06c5-609b-4ede-99f9-e24d823bffcb None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:32.848201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 450/902] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:32.900306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:32.902244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:32.902939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:32.903694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:32.905247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:32.905994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:32.908979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:32.932633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 453/903] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:32.940407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:33.016048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:33.051188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97322d47-dfa2-4e9b-9c81-47513bbdc0ba None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:33.091820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:33.092644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:33.093028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:33.093400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:33.094323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] RBAC: Adding query filter params (role.id=b4e1d70932c5441e8cbecc0f7b1d41ad, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=589534f84a6143a2a194ccbd1c84373f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:33.094731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:33.096273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:33.102612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 451/904] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:32 2022] GET /identity/v3/role_assignments?user.id=589534f84a6143a2a194ccbd1c84373f&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=b4e1d70932c5441e8cbecc0f7b1d41ad => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:33.132453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:33.154616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:33.257900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b87481-59c0-4e84-8815-d7f7d4b9f245 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:33.296146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:33.296877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:33.297157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:33.297432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:33.298027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:33.299394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:33.309303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:33.310478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:33.311571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 454/905] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 31 00:16:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:33.316647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:33.443915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55392d5-535e-4a7e-aec6-94f098181609 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:33.499298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:33.499907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:33.500184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:33.500485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:33.501119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:33.501418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:33.502849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:33.503938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:33.508583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 452/906] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:33.518160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:33.651002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:33.683731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a447f7e-6f17-4114-b811-c3d13e3320b5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:33.746420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:33.747441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:33.747924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:33.748441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:33.749569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:33.750187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:33.752732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:33.768489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 455/907] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:33.773135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:33.952815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6887-a048-4342-8082-7fd8effebc8a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:33.992592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:33.993646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:33.994142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:33.994744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:33.995898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] RBAC: Adding query filter params (interface=public, service_id=2b8e63dd7b0d43f6a9dbafac3770aaf2, region_id=RegionOne) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:33.996463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:33.998062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630d8d8-7be2-48aa-97a7-f3daa3b7e5d4 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:34.005428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 453/908] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:33 2022] GET /identity/v3/endpoints?service_id=2b8e63dd7b0d43f6a9dbafac3770aaf2&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:34.011798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:34.141267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9970-4c8c-43cb-bccc-b74794addde7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:34.177693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc75e6-8fc6-4975-93f6-e52bae880edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:34.178366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc75e6-8fc6-4975-93f6-e52bae880edb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:34.178641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc75e6-8fc6-4975-93f6-e52bae880edb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:34.178907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc75e6-8fc6-4975-93f6-e52bae880edb None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:34.179505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc75e6-8fc6-4975-93f6-e52bae880edb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:34.180895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc75e6-8fc6-4975-93f6-e52bae880edb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:34.188721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 456/909] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:34 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:16:34.422648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132c02-78b0-4705-aeb6-ee3f1602486e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:34.423306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132c02-78b0-4705-aeb6-ee3f1602486e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:34.423789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132c02-78b0-4705-aeb6-ee3f1602486e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:34.425783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 454/910] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:34.431512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be03e7-1ecd-4c3a-bec9-ed01fc401f5d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:34.431793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be03e7-1ecd-4c3a-bec9-ed01fc401f5d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:34.432059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be03e7-1ecd-4c3a-bec9-ed01fc401f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:34.452035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f1be03e7-1ecd-4c3a-bec9-ed01fc401f5d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:34.453333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f1be03e7-1ecd-4c3a-bec9-ed01fc401f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:34.455461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:34.538714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 457/911] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:34.829711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132c02-78b0-4705-aeb6-ee3f1602486e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:34.904670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:34.931327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9411-a55b-47aa-b551-7daa88e953a5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:34.931846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9411-a55b-47aa-b551-7daa88e953a5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:34.931846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9411-a55b-47aa-b551-7daa88e953a5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:34.933734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 458/912] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:34.938918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de592a15-28ad-40e7-979f-9ce446e49568 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:34.939197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de592a15-28ad-40e7-979f-9ce446e49568 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:34.939467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de592a15-28ad-40e7-979f-9ce446e49568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:34.945510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132c02-78b0-4705-aeb6-ee3f1602486e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:34.958863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-de592a15-28ad-40e7-979f-9ce446e49568 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:34.960062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-de592a15-28ad-40e7-979f-9ce446e49568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:34.981765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662e184-58d7-43fc-a2b5-8b880b437134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:34.982370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662e184-58d7-43fc-a2b5-8b880b437134 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:34.982747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662e184-58d7-43fc-a2b5-8b880b437134 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:34.983018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662e184-58d7-43fc-a2b5-8b880b437134 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:34.983682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662e184-58d7-43fc-a2b5-8b880b437134 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:34.985077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662e184-58d7-43fc-a2b5-8b880b437134 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:35.033920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 459/913] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:35.092271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 455/914] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:34 2022] POST /identity/v3/services => generated 241 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:35.233885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e783a-333e-4a49-8e90-2ae6c2cf33c2 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:35.234607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e783a-333e-4a49-8e90-2ae6c2cf33c2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:35.235205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e783a-333e-4a49-8e90-2ae6c2cf33c2 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:35.238090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 460/915] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:35.246948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:35.247639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:35.248224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:35.273559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:35.274761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:35.359231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 456/916] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:35.400055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e783a-333e-4a49-8e90-2ae6c2cf33c2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:35.488146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:35.527349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e783a-333e-4a49-8e90-2ae6c2cf33c2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:35.564115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:35.564784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:35.565064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:35.565337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:35.566180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:35.567555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:35.577590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:35.578613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:35.580004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 461/917] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:35.584591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:35.645881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:35.676913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:35.717564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1905f-bfeb-46d9-9b4f-a575832dd898 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:35.764932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:35.766118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:35.766730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:35.767227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:35.768085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:35.768632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:35.770562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:35.778126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 457/918] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:35.786293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:35.841909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f54506-ea3b-44a0-afdb-a406a257c961 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:35.879495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:35.885252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:35.885995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:35.886276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:35.886551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:35.899192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:35.905026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:35.906022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:35.906894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 462/919] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 00:16:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:35.913506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:35.918918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ebb3e-7d42-42b9-9909-55de0ac7ca62 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:35.934575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:35.959026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:35.959979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:35.960475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:35.960898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:35.961723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:35.962191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:35.964429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:35.971178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 458/920] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 00:16:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:35.977764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:36.038243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a285b-3045-4a78-a185-2e5eb834b811 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:36.084201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:36.085615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.086088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.086620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.087456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:36.088043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:36.089488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:36.101139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 463/921] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:16:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:36.107987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:36.184480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dab312-988b-4edc-ba32-f5f74d3d5f79 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:36.189167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:36.249322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099a6ef-d883-4ef1-94dd-68b78330b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:36.250054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099a6ef-d883-4ef1-94dd-68b78330b8cb None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.250431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099a6ef-d883-4ef1-94dd-68b78330b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.250675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099a6ef-d883-4ef1-94dd-68b78330b8cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.251478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099a6ef-d883-4ef1-94dd-68b78330b8cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:36.253020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099a6ef-d883-4ef1-94dd-68b78330b8cb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:36.256814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a3be-4db1-4a7a-b44c-156f08d9fba1 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:36.299026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:36.300075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.300606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.301084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.324161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:36.326555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-010387e3-6ad0-4e63-8296-1a2b7512895c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:36.355073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:36.356175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:36.357431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:36.357431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:36.357431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:36.357431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 464/922] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:36.357676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87778b68-7830-422f-9f9f-8c79cdee7b1e None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.357743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87778b68-7830-422f-9f9f-8c79cdee7b1e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.358145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87778b68-7830-422f-9f9f-8c79cdee7b1e None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.359345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 465/923] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:36.362209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87778b68-7830-422f-9f9f-8c79cdee7b1e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:36.410097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 459/924] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:36.411743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.412085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.412483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.446801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:36.449867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:36.496502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:36.542169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87778b68-7830-422f-9f9f-8c79cdee7b1e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:36.544062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:36.571046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 460/925] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:36.590225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:36.592538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.592929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.593247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.594017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:36.594331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:36.595772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:36.611660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 466/926] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:36.617683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:36.743280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921643d7-656f-48a9-9278-a728ac149ce1 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:36.785289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:36.786338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:36.786817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:36.787281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:36.796541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:36.798308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:36.812545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:36.813570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:36.814663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:36.817599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 461/927] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:16:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:36.826572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:36.860885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:36.894718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:36.957629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc6e99-a4a5-4567-b722-2ead92d56edd None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.000570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2728c-c01a-484d-96ea-e9f301c5ee79 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.006170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.007140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.007621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.008089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.009173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:37.009693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.012109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.034013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 467/928] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 00:16:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:37.040011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.045723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.046103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.046485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.046857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.047551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.049122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:37.059075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:37.060255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:16:37.061330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:37.063474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 462/929] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 31 00:16:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:37.072375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.075773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:37.179329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6e035-a9ff-45f0-b780-d7f199fb0898 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.204076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:37.224476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f4c75-f5a2-4133-9613-641a4f1e0853 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.229387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.230528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.230995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.231414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.248104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.250546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.267957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.268719 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.269124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.269524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.270299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:37.270764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-5d39114c-aac4-4290-ac10-aa3113994d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:37.271483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:37.273468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 468/930] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ebdd0-f63a-434a-a29d-e2f2d162a51d None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ebdd0-f63a-434a-a29d-e2f2d162a51d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.275198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ebdd0-f63a-434a-a29d-e2f2d162a51d None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.279485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf4f5-10fb-4ed7-9823-700a208c3826 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.280524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 469/931] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:37.281218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ebdd0-f63a-434a-a29d-e2f2d162a51d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.295371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 463/932] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:37.295371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.295371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.295371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.315797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:37.317225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:37.365661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:37.399778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 464/933] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:37.402135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ebdd0-f63a-434a-a29d-e2f2d162a51d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.405835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.444641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.445127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.445525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.445896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.446665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:37.447060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.449521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.474506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 470/934] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:37.475849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.589183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:37.601695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bbf92-babf-4da4-925f-5b40e41a8c00 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.624858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61062ced-5b0e-438a-bc97-e3010101a69f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.641008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.641790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.642190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.642575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.650068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.651758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.652718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:37.666295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:37.674163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:37.676133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 471/935] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:37.682338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.700234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.701237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.701602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.701946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.703268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:37.703741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.705940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.716403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 465/936] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:37.719804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.826110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9b35a-c49f-4a26-8c9b-797cf4d27cae None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.852179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d202c0-0d85-44fd-b35b-d015631069e1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:37.864949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.865555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.865832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.866104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.866790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:37.867097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.868612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.881737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 472/937] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:37.882615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:37.893919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:37.894625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:37.894903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:37.895174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:37.895856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:37.897264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:37.944574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:37.992241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ec3ec-1bc1-4fb5-959e-9c31d7278a8d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:38.027860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b6422-652c-476e-8232-90785efdef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:38.028519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b6422-652c-476e-8232-90785efdef26 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.028799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b6422-652c-476e-8232-90785efdef26 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.029071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b6422-652c-476e-8232-90785efdef26 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.029648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b6422-652c-476e-8232-90785efdef26 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:38.031009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b6422-652c-476e-8232-90785efdef26 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-151b6422-652c-476e-8232-90785efdef26 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:38.039762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:38.042532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:38.043428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 473/938] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:38.044235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b6422-652c-476e-8232-90785efdef26 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:38.062901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:38.069750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 466/939] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:16:38.070687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:38.209120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b6422-652c-476e-8232-90785efdef26 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:38.226067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb25e-dfe0-44d6-9137-c0b8d635d147 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:38.235238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:38.258160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:38.259138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.259636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.259908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/589534f84a6143a2a194ccbd1c84373f/roles/b4e1d70932c5441e8cbecc0f7b1d41ad` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.262684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=589534f84a6143a2a194ccbd1c84373f, role_id=b4e1d70932c5441e8cbecc0f7b1d41ad)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:38.264129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:38.272313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:38.273031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.273313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.273578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.274208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:38.274503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:38.275868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:38.282530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 467/940] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:38.288001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:38.396708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570c21-b275-47fd-81ae-edcad79e589d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:38.429047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 474/941] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:16:38 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/589534f84a6143a2a194ccbd1c84373f/roles/b4e1d70932c5441e8cbecc0f7b1d41ad => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:16:38.465924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:38.466807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.467237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.467655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.468495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:38.475644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:38.475644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b87d0-ad72-4ef8-a53e-1f993978bbef None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:38.565263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:38.570232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 468/942] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:38.578433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:38.600224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0087-cb82-40ae-aca9-e98ac60beda3 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.600679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0087-cb82-40ae-aca9-e98ac60beda3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.600990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0087-cb82-40ae-aca9-e98ac60beda3 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.602674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 469/943] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:38.608296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.608711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.609052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.638418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:38.639968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:38.735687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 470/944] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:38.743491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2bc1f-ee6d-46e9-bbd9-ca72b62ab9fb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:38.782862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:38.783723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.784172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.784618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.785497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] RBAC: Adding query filter params (interface=public, service_id=ee15feabeed946a7a18b119d2317dd1a, region_id=RegionOne) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:38.785951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:38.787452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:38.794107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 475/945] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:38 2022] GET /identity/v3/endpoints?service_id=ee15feabeed946a7a18b119d2317dd1a&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:38.799932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:38.864992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:38.912506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebc38d-9db6-42e8-9a84-e412b6a382e4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:38.951486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:38.952107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:38.952417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:38.952694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:38.953310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:38.954744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:38.961484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 471/946] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:38 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:16:39.011938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:39.063631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:39.144224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088430d-ca8e-495a-9f1a-4f714565b70f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:39.181182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:39.181794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:39.182068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:39.182441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:39.183343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:39.184877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:39.187820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING keystone.server.flask.common [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 00:16:39.238820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.sql.core [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 00:16:39.238820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 00:16:39.238820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [parameters: {'id': 'e47ef396f7a544b88341e21ab99a07c2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 00:16:39.238820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88219) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e47ef396f7a544b88341e21ab99a07c2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e47ef396f7a544b88341e21ab99a07c2. Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:16:39.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application result.read() Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 00:16:39.258288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application yield resource Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise self.value Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application session.commit() Aug 31 00:16:39.259244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self.session.flush() Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application compat.raise_( Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 00:16:39.260180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 00:16:39.261992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application result.read() Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:16:39.262831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [parameters: {'id': 'e47ef396f7a544b88341e21ab99a07c2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:39.263708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:39.265920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e47ef396f7a544b88341e21ab99a07c2. Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:39.268169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 476/947] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 31 00:16:39 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:39.269559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:39.381952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf59e-adcc-41db-a3a7-98b4756eb491 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:39.404713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:39.420100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:39.428602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:39.428602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:39.428602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:39.429047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:39.430525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e6378e0f-b3b5-49d1-81f1-c6ef927b0f4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:39.444520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:39.445639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:39.446732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:39.450146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 472/948] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:39.458856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:39.578804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:39.585259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4576c-21f4-4e66-a29c-905a6dbb4e96 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:39.629302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:39.630206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:39.630558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:39.630834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:39.631953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:39.631953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:39.634205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ff88f-dd62-4971-b775-6e73c409660e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:39.668754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 477/949] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:40.451962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad576d-73a8-464c-94cf-ed158da6088e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:40.453105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad576d-73a8-464c-94cf-ed158da6088e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:40.453622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad576d-73a8-464c-94cf-ed158da6088e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:40.455485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 473/950] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:40.461335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafada-451c-42f5-a2f2-62066569616b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:40.462230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafada-451c-42f5-a2f2-62066569616b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:40.462734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aafada-451c-42f5-a2f2-62066569616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:40.493002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-21aafada-451c-42f5-a2f2-62066569616b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:40.495391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-21aafada-451c-42f5-a2f2-62066569616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:40.547773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1054b-8510-49ac-b402-f3615d343087 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:40.548252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1054b-8510-49ac-b402-f3615d343087 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:40.548414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1054b-8510-49ac-b402-f3615d343087 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:40.551052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 474/951] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:40.555272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:40.555552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:40.555834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:40.578445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:40.579475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 478/952] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:40.579738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:40.667158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 475/953] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:40.859730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafada-451c-42f5-a2f2-62066569616b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:40.961995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:40.989739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:41.011702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aafada-451c-42f5-a2f2-62066569616b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.065966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.066749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.067758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.067758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.067758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.069188 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:41.081237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:41.082394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:41.083287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.087941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 479/954] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:41.089432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.120564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaced2-7785-4e50-8653-a8fd0a270408 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.138168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:41.173473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef041b8c-3fdd-4393-9da3-44e31eaa7cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.174731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef041b8c-3fdd-4393-9da3-44e31eaa7cf8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.175026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef041b8c-3fdd-4393-9da3-44e31eaa7cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.175308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef041b8c-3fdd-4393-9da3-44e31eaa7cf8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.191914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef041b8c-3fdd-4393-9da3-44e31eaa7cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-ef041b8c-3fdd-4393-9da3-44e31eaa7cf8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:41.196751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:41.204620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 476/955] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 00:16:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:41.205500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdaa12-7472-411f-b349-bb0497ee5cae None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.206288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdaa12-7472-411f-b349-bb0497ee5cae None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.206288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdaa12-7472-411f-b349-bb0497ee5cae None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.206288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 477/956] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:41.206288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdaa12-7472-411f-b349-bb0497ee5cae None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.253123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ea891-d148-4d39-89c7-6bedb09a9df7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.293891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.294504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.294777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.295050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.295677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:41.295975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.297431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50965b-8f82-4bc3-b663-9cc56e6f80ea None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.302361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 480/957] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:41.303543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.303827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.304098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.325361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:41.326581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:41.339698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:41.344987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfdaa12-7472-411f-b349-bb0497ee5cae None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.387154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.387661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.387940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.388209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.388911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:41.389209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.390589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.405172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 478/958] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:41.406032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.440700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 481/959] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:41.442584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.468134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:41.559163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35014fb6-92ac-407b-9489-93a4d2157444 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.579759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186249a-c654-4394-87ce-623aeee24fa6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.599143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.599762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.600044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.600314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.601115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:41.601426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.602816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.608440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 479/960] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:41.623161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.623819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.624097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.628506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.649225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.651706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.656369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:41.678525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:41.679748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:41.680963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:41.680963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:41.680963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.682934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 482/961] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:41.689239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.752049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.819688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:41.831599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264e608-91be-481f-ab38-0cf6afeee443 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.888676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.889902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.890457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.891014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.892252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:41.892873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.895414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.919025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 480/962] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:41.920883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f548651-48c1-454f-ab12-9d69b39e51fa None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:41.927795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:41.959666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:41.960674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.961245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.961710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.962501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:41.965103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-8861bb92-7e84-4f5b-920c-79f7a6f24f29 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:41.975250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:41.976305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:41.977187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:41.980962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 483/963] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:41.985806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1695-feba-4379-89e9-6db36b052719 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:41.986491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1695-feba-4379-89e9-6db36b052719 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:41.986769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1695-feba-4379-89e9-6db36b052719 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:41.989398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 484/964] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:41.990382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc1695-feba-4379-89e9-6db36b052719 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.068987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:42.094134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdfaee-5e19-4360-9705-0410a6943011 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:42.127814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc1695-feba-4379-89e9-6db36b052719 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:42.134705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:42.135982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.136150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.136513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.145305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:42.146771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-48fc989a-98d9-43cb-96ff-f32ec82d9293 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:42.160097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:42.161252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:42.162292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 481/965] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:42.162923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.163001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.163791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.177334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:42.177878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.178206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.178461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.179025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:42.179319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:42.180932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:42.183932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:42.185833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:42.192717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 485/966] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:42.192717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.209848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:42.296513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 482/967] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:42.297772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.338950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383b708-6aa9-4c4e-a39a-9a2326a04f2b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:42.374661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:42.394405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:42.395427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.396050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.396575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.398146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:42.398576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:42.401177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:42.433943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 486/968] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 00:16:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:42.440525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.454094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ac54-a720-4139-80a1-d6911ba528fc None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:42.500714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:42.501462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.501659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.502002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.502834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:42.503244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:42.504998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:42.511942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 483/969] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:42.525279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.544747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:42.575146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98fa7-349d-4218-9d2c-66364a315e05 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:42.616000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:42.616675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.616986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.617277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.666021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c055f-9c0c-4a96-b0d4-4ca2fd5deca8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:42.687530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:42.711341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:42.712030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:42.712313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:42.712602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:42.713495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:42.714885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:42.715146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:42.716684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:42.846091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 484/970] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:42.857384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.865412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 487/971] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:16:42.866916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:42.961416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:42.981133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:43.005399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c5294-0eb0-45ad-b874-b73758b1e455 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.033145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029778a-e343-41e5-a8dd-82ebe43d5651 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.065820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.066766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.067052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.067317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.079365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b63f8-650d-4968-9244-72912033e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.080929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b63f8-650d-4968-9244-72912033e79f None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.080929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b63f8-650d-4968-9244-72912033e79f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.080929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b63f8-650d-4968-9244-72912033e79f None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.081204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b63f8-650d-4968-9244-72912033e79f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.082647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b63f8-650d-4968-9244-72912033e79f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.084139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.086716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.086830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING keystone.server.flask.common [None req-033b63f8-650d-4968-9244-72912033e79f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:43.111107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:43.112181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:43.113246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:43.113246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:43.113246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:43.116398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 488/972] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:43.117968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:43.128560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.sql.core [None req-033b63f8-650d-4968-9244-72912033e79f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 00:16:43.128560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 00:16:43.128560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [parameters: {'user_id': 'fb90b3d04c184e9c9e8ca8714921c164', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 00:16:43.128560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88218) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-033b63f8-650d-4968-9244-72912033e79f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result.read() Aug 31 00:16:43.140720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:16:43.141647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application yield resource Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise self.value Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application session.commit() Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 00:16:43.142640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.session.flush() Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application compat.raise_( Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 00:16:43.143545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:16:43.144475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application result.read() Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:16:43.145916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fb90b3d04c184e9c9e8ca8714921c164', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:43.148390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 00:16:43.151766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 485/973] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 00:16:42 2022] POST /identity/v3/users => generated 160 bytes in 299 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:43.154360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b63f8-650d-4968-9244-72912033e79f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:43.283375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d253-322a-4c99-8a39-c3f09d72338b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.294246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b63f8-650d-4968-9244-72912033e79f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.328426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:43.332273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.333375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.333870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.334647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.335787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:43.336298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.338116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.340680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:43.343939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.344728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.351265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.351265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.354722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.356557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.360079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 489/974] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:43.362059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:43.377701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:43.378754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:43.379718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 486/975] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:16:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:43.384666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:43.499928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d8c0e-d969-49db-95ed-700c69db91b7 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.522097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8f8de-b1c6-4027-97d8-55f07252da3e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.549094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.549721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.550034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.550958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.561585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.563991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.568182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.569217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.569497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.569771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.570542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:43.570847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.572267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:43.577108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:43.579111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:43.581284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:43.588356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 490/976] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:43.588356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:43.628779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 487/977] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 00:16:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:43.660552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:43.714669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fd473-08a1-4768-841a-64d8892233d0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.753606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.754228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.754509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.754780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.755486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:43.755791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.757347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba7a53-5e15-4599-9d4e-e22b07a9c4b1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:43.776421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 491/978] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:43.779113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:43.835067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd4f8-0671-4a2c-bb01-a54d552d6b92 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.835615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd4f8-0671-4a2c-bb01-a54d552d6b92 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.835615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd4f8-0671-4a2c-bb01-a54d552d6b92 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.837382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 492/979] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:43.844170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.844758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.845026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.847485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:43.865068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:43.866744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:43.942634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcf25b-ed1d-4aa7-bc3e-4531ec1851b1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:43.964799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 493/980] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:43.984071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:43.984847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:43.985134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:43.985410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:43.986607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] RBAC: Adding query filter params (role.id=b4e1d70932c5441e8cbecc0f7b1d41ad, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=589534f84a6143a2a194ccbd1c84373f) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:43.986938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:43.988375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08ee6f-5cd8-4185-82a5-23347e134028 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:44.000403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 488/981] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:43 2022] GET /identity/v3/role_assignments?user.id=589534f84a6143a2a194ccbd1c84373f&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=b4e1d70932c5441e8cbecc0f7b1d41ad => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:44.237943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:44.299451 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:44.380365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1210c-d16b-436f-b57b-8cc0829da456 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:44.436516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41572f7-ad4b-4f24-9a96-43638f23b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:44.437151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41572f7-ad4b-4f24-9a96-43638f23b4d1 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:44.437436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41572f7-ad4b-4f24-9a96-43638f23b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:44.437814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41572f7-ad4b-4f24-9a96-43638f23b4d1 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:44.438662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41572f7-ad4b-4f24-9a96-43638f23b4d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:44.441445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41572f7-ad4b-4f24-9a96-43638f23b4d1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:44.523331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 494/982] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:44 2022] POST /identity/v3/services => generated 242 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:45.097704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76c92-628a-4d3f-b9dd-ac4a041e6815 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:45.098489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76c92-628a-4d3f-b9dd-ac4a041e6815 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:45.098489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76c92-628a-4d3f-b9dd-ac4a041e6815 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:45.099955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 489/983] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:45.105703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:45.108598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:45.108598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:45.126790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:45.128031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:45.254199 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 495/984] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:45.554664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c76c92-628a-4d3f-b9dd-ac4a041e6815 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:45.701994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c76c92-628a-4d3f-b9dd-ac4a041e6815 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:45.706580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:45.747280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:45.748360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:45.749087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:45.749585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:45.750391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:45.751956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:45.762275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:45.763355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:45.764193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:45.768374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 490/985] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:45.772844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:45.807942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:45.901485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18ec82-e268-44fe-8298-8e81b82324a3 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:45.947391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:45.948127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:45.948426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:45.948694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:45.949327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:45.949620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:45.951039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd84d4-0788-48be-a711-70c5c393c640 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:45.960835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 496/986] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:45.964687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:46.106420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44470f8-6e1c-4cbc-9549-66a8dd099367 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:46.117396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2366ac-32a6-4feb-bc4c-5bd14d7f12b5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.117923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2366ac-32a6-4feb-bc4c-5bd14d7f12b5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.118155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2366ac-32a6-4feb-bc4c-5bd14d7f12b5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.120907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 497/987] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:46.128849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.129305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.129585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.148525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:46.149796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:46.150131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:46.150817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.151310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.151883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.152368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:46.152679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:46.155224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:46.160944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 491/988] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:46.167901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:46.221223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:46.250749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 498/989] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:46.311496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01fb3-bd58-449e-a906-898c318554bd None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:46.354862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:46.355521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.355774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.356001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.356797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] RBAC: Adding query filter params (interface=admin, service_id=ee15feabeed946a7a18b119d2317dd1a, region_id=RegionOne) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:46.357047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:46.358446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:46.364633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 492/990] 10.222.0.65 () {56 vars in 1249 bytes} [Wed Aug 31 00:16:46 2022] GET /identity/v3/endpoints?service_id=ee15feabeed946a7a18b119d2317dd1a&interface=admin®ion_id=RegionOne => generated 193 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:46.371225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:46.402993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:46.505999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503cfa8-7ffa-4fe5-81c4-7a398a830afa None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:46.533137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:46.551409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abda74-16f1-43af-a09c-8fa779475b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:46.552036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abda74-16f1-43af-a09c-8fa779475b78 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.552370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abda74-16f1-43af-a09c-8fa779475b78 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.552671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abda74-16f1-43af-a09c-8fa779475b78 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.553189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abda74-16f1-43af-a09c-8fa779475b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:46.554554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abda74-16f1-43af-a09c-8fa779475b78 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:46.568357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 499/991] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:46 2022] GET /identity/v3/services => generated 1727 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:16:46.663643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:46.670983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a29cc2-ecb9-4fad-aeea-2eabe9ac75e3 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:46.676623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e7d96-bcc6-4616-9476-56ed34b51799 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.676623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e7d96-bcc6-4616-9476-56ed34b51799 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.676623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e7d96-bcc6-4616-9476-56ed34b51799 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.676623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 500/992] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:46.678850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.679147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.679422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.704170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:46.704170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:46.719325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:46.719969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.720247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.720607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] PATH_INFO: `/v3/services/image` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.721195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:46.722575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:46.732696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:46.734921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:46.737037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 493/993] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 31 00:16:46 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:46.740199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:46.810863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 501/994] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:46.886595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1bff-4de1-426c-a972-5d80d4a00f87 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:46.935593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:46.936094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:46.936402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:46.936674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:46.937309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] RBAC: Adding query filter params (name=image) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:46.937613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:46.939003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:46.949521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 494/995] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:46.955542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:46.981945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:47.092215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4559ccd-11ac-46d5-afdc-fe45cf2a7579 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:47.106881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:47.134054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:47.136108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:47.136574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:47.137105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:47.137832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] RBAC: Adding query filter params (type=image) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:47.138852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:47.140299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:47.141893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:47.156357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 502/996] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:47.264310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d463b-444a-4c41-bcdf-0c391a4d8b42 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:47.324022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:47.324835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:47.325109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:47.325382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:47.325957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:47.327332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:47.336770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:47.337787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:16:47.338698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 495/997] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:47.344307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:47.456482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:47.483900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3882-fa12-428a-bbfd-b287d20044bd None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:47.534159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:47.534862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:47.535304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:47.535697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:47.536509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:47.536941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:47.538471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:47.544930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 503/998] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:47.551723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:47.660665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:47.674441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206596-efa9-4d70-b403-159ef0a2ef68 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:47.719869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:47.720600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:47.720880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:47.721168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:47.721821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:47.722121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:47.723491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:47.729895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 496/999] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:47.735658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:47.857920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9365-2678-4ec2-9241-4c54e7b5b7fe None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:47.896873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:47.897790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:47.898243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:47.898691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:47.899908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] RBAC: Adding query filter params (interface=public, service_id=f06770659aa34d59bdc9de522085320f, region_id=RegionOne) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:47.900935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:47.903434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce7070-4dd2-4f8c-86cf-28887c5b4512 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:47.915385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 504/1000] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:47 2022] GET /identity/v3/endpoints?service_id=f06770659aa34d59bdc9de522085320f&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:47.917410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:48.027238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a86038-5c7f-45e3-8683-9140b64e9a4d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:48.063258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eeb9a3-3549-4752-8e3b-36d10fb93471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:48.063861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb9a3-3549-4752-8e3b-36d10fb93471 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:48.064133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb9a3-3549-4752-8e3b-36d10fb93471 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:48.064426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb9a3-3549-4752-8e3b-36d10fb93471 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:48.065278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eeb9a3-3549-4752-8e3b-36d10fb93471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:48.066643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eeb9a3-3549-4752-8e3b-36d10fb93471 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:48.078034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 497/1001] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:47 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:16:48.678585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93976-a090-48d3-a8de-b1b504531d50 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:48.679579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93976-a090-48d3-a8de-b1b504531d50 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:48.680062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93976-a090-48d3-a8de-b1b504531d50 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:48.682432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 505/1002] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:48.689500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57d138-65b1-4c6f-a589-3b3e80ab2aa2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:48.691894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57d138-65b1-4c6f-a589-3b3e80ab2aa2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:48.693050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57d138-65b1-4c6f-a589-3b3e80ab2aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:48.717125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8f57d138-65b1-4c6f-a589-3b3e80ab2aa2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:48.719024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8f57d138-65b1-4c6f-a589-3b3e80ab2aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:48.734602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:48.805649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 498/1003] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:49.107261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a93976-a090-48d3-a8de-b1b504531d50 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:49.122955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:49.185877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029361-6cfc-41c9-868b-852b32353858 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:49.186169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029361-6cfc-41c9-868b-852b32353858 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:49.186302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029361-6cfc-41c9-868b-852b32353858 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:49.187762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 499/1004] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:49.193249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:49.193530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:49.193807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:49.217658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:49.218903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:49.254991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a93976-a090-48d3-a8de-b1b504531d50 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:49.302289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 500/1005] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:49.307831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:49.308791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:49.309252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:49.309706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:49.310465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:49.312000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:49.325648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:49.326678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:16:49.327648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:49.329769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 506/1006] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 00:16:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:49.335815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:49.437715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:49.477173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fb28f-28fe-4657-865a-04901c2cfd69 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:49.533324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:49.533949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:49.534230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:49.534505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:49.535221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:49.535453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:49.536921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:49.550231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 501/1007] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:49.552156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:49.592624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:49.668698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:49.715713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248578a8-880e-495c-b991-d89d9c242906 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:49.760518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d10d9-fe05-4a17-ade4-050d48eb8201 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:49.770999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:49.771998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:49.772414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:49.773012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:49.774550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:49.775189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:49.778047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:49.792400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 507/1008] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 00:16:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:49.796441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:49.834147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f425fb-01f2-48be-85a0-3e083a3b687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:49.834884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f425fb-01f2-48be-85a0-3e083a3b687c None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:49.835167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f425fb-01f2-48be-85a0-3e083a3b687c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:49.835437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f425fb-01f2-48be-85a0-3e083a3b687c None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:49.836593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f425fb-01f2-48be-85a0-3e083a3b687c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:49.839569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f425fb-01f2-48be-85a0-3e083a3b687c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:49.921728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:49.931669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 502/1009] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:49 2022] POST /identity/v3/services => generated 242 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:16:49.976300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fa180-0e35-4bc8-92b8-857aac16d8fa None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:50.055525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:50.056382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:50.057369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:50.057823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:50.059237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:50.061581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:50.076861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:50.159263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cb420-3944-41c2-a64c-025759de1b03 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:50.159538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cb420-3944-41c2-a64c-025759de1b03 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:50.159805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cb420-3944-41c2-a64c-025759de1b03 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:50.161241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 503/1010] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:50.166762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:50.167071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:50.167315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:50.192714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:50.194037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:50.201359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 508/1011] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:50.334169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 504/1012] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:50.611542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:50.748818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f6a00-cf25-43d0-acf2-ce9e59ed960f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:50.787114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:50.787750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:50.788022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:50.788284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:50.788902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:50.790272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:50.800697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:50.802169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:50.803493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 509/1013] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:50.808454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:50.954559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c86e4-1249-48ec-9b9d-c0ff8f1262f2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:50.990523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:51.002811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:51.003637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:51.004168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:51.004599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:51.005517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] RBAC: Adding query filter params (name=compute) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:51.005903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:51.007613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:51.014803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 505/1014] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:51.021731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:51.122600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:51.196910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bd360-3620-453a-82af-80776bd9f4c8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:51.265344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:51.266370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:51.266864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:51.267344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:51.268526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:51.269069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:51.271651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20d2e-6bde-4d2e-bfd8-8efe8baca850 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:51.285336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 510/1015] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 00:16:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:51.286595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:51.437099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb448b-bc65-4240-9b72-c1049683acda None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:51.489070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8827fd5-0baf-49ca-9770-7fd361685cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:51.489691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8827fd5-0baf-49ca-9770-7fd361685cd0 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:51.489969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8827fd5-0baf-49ca-9770-7fd361685cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:51.490239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8827fd5-0baf-49ca-9770-7fd361685cd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:51.490923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8827fd5-0baf-49ca-9770-7fd361685cd0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:51.492450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8827fd5-0baf-49ca-9770-7fd361685cd0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:51.574480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 506/1016] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:16:52.245512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6501537-212c-49fb-8cdb-0e2eb6c5f07a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:52.246928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6501537-212c-49fb-8cdb-0e2eb6c5f07a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:52.247418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6501537-212c-49fb-8cdb-0e2eb6c5f07a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:52.250091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 511/1017] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:52.259354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf235c-f754-48eb-804e-9b494857b117 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:52.260273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf235c-f754-48eb-804e-9b494857b117 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:52.260273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf235c-f754-48eb-804e-9b494857b117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:52.292274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-aabf235c-f754-48eb-804e-9b494857b117 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:52.295147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-aabf235c-f754-48eb-804e-9b494857b117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:52.356672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:52.398427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 507/1018] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:52.526438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b330e-012c-45b8-b69f-5707d2fa48ee None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:52.527116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b330e-012c-45b8-b69f-5707d2fa48ee None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:52.527612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b330e-012c-45b8-b69f-5707d2fa48ee None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:52.530415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 512/1019] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:52.536739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:52.537037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:52.537378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:52.567076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:52.569246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:52.704001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 508/1020] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:52.806306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b330e-012c-45b8-b69f-5707d2fa48ee None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:52.982573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:52.986991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b330e-012c-45b8-b69f-5707d2fa48ee None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.030376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.051477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.052591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.053072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.053597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] PATH_INFO: `/v3/services/image` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.054664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.057381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:53.076840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:53.078854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:53.082101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 513/1021] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 31 00:16:52 2022] GET /identity/v3/services/image => generated 86 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:53.088826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.195375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e51b-c672-4bf0-b086-20f39ef4588a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.199838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.269993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.272357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.273111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.273960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.276672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.278425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a0bb2-a8a0-4e13-9f16-4c591e83b483 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.279766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.283388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 509/1022] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:53.289464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.334010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.334885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.335357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.335813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.336918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] RBAC: Adding query filter params (name=image) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:53.337446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.340543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.349377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 514/1023] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:53.359457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.449267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.487586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee8ec-51a8-450e-9ea9-d4169e968d72 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.521909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26f00-5f33-4a9b-8ff9-d32cd18cc4cb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.526269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.526878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.527152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.527490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.528552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.529596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.561179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.561807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.562080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.562376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.562958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] RBAC: Adding query filter params (type=image) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:53.563254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.564638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.570212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 515/1024] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:53.576688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.622065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 510/1025] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:16:53 2022] POST /identity/v3/projects => generated 311 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:53.755133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38a5b-050b-4f63-ab31-9022d4b78615 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.779153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.802882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.803495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.803770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.804035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.804763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] RBAC: Adding query filter params (interface=public, service_id=a25061ee8c654467bd27f3a996167323, region_id=RegionOne) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:53.805065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.806470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2994f-fbfa-4b7c-a2cb-4c29ffebebe7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.813197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 516/1026] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:16:53 2022] GET /identity/v3/endpoints?service_id=a25061ee8c654467bd27f3a996167323&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:53.819860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.861795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.979840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b653e-ce40-49c2-8334-f27205cc3d29 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:54.033579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c10c6-d83b-4503-9c02-1bf0862a2381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:54.034621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c10c6-d83b-4503-9c02-1bf0862a2381 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.035088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c10c6-d83b-4503-9c02-1bf0862a2381 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.035551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c10c6-d83b-4503-9c02-1bf0862a2381 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.036630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c10c6-d83b-4503-9c02-1bf0862a2381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:54.039062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c10c6-d83b-4503-9c02-1bf0862a2381 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:54.051932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 511/1027] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:16:53 2022] GET /identity/v3/services => generated 1957 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:16:54.329087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e063f1-79f9-428f-8673-1271c6134930 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.329616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e063f1-79f9-428f-8673-1271c6134930 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.330012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e063f1-79f9-428f-8673-1271c6134930 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.332515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 517/1028] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:54.340987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.341712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.342044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.370431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:54.372403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:54.496373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 512/1029] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:54.984872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e063f1-79f9-428f-8673-1271c6134930 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.155186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e063f1-79f9-428f-8673-1271c6134930 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:55.222580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:55.223622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.224104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.224617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.246761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:16:55.255754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:55.257618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:55.259200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 518/1030] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 00:16:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:55.269990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.300791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.435012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbc453-7894-40b9-aa34-222dc656d4f8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:55.497464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:55.498399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.498882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.499354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.500564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:55.501097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:55.503625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:55.524056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 513/1031] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:16:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:55.526877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.579412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.685803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9208361-0205-46c7-bd76-1948ae8cd583 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:55.745231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:55.746281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.746721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.747207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.766614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:55.768942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:55.795999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:55.798227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:55.800070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:55.800070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:55.800070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:55.801229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 519/1032] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:55.807768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.948723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.974073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8deab7-d097-4bf8-9077-e313491d9456 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.034396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.035365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.035869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.036391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.037605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:56.038135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.040806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dcc10-8a2b-4fba-866d-b88ad7e2f5dd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.058258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 514/1033] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:56.065373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:56.190800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226036bf-c9d3-4112-9129-57b621f7d197 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.191127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226036bf-c9d3-4112-9129-57b621f7d197 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.191349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226036bf-c9d3-4112-9129-57b621f7d197 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.193196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 515/1034] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:16:56.201419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d284f-a580-4d60-92b4-8ce69d648095 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.201853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d284f-a580-4d60-92b4-8ce69d648095 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.202297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d284f-a580-4d60-92b4-8ce69d648095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.220171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:56.236222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77aafc-2e06-41a2-8323-1f2453a951fb None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.238888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-001d284f-a580-4d60-92b4-8ce69d648095 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:56.240297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-001d284f-a580-4d60-92b4-8ce69d648095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:56.288511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.289043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.289373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.289685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.302248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.304891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-d4dd4621-9436-4a14-9411-b2119d55c1eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:16:56.319542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:56.321491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:16:56.325398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:56.326817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 520/1035] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:16:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:56.327647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43732c-ab97-48fb-8377-f1b592a1876f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.327927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43732c-ab97-48fb-8377-f1b592a1876f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.328193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43732c-ab97-48fb-8377-f1b592a1876f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.329564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 521/1036] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:56.330261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43732c-ab97-48fb-8377-f1b592a1876f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:56.356382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 516/1037] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:16:56.356382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.356382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.356382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.389157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:56.390939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:56.495219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:56.508375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 517/1038] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:16:56.520147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43732c-ab97-48fb-8377-f1b592a1876f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.559665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.560906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.561044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.561416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.562422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:56.562859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.564473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.595249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 522/1039] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:16:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:56.597591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:56.717435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:56.729830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:56.737065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf59ab1-3890-40e7-a928-94e872273f20 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.796380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.797219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.797386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.797542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.865720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38a3d-be11-4d4d-9919-2f7adbd956d8 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.877193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:56.880989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.883306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.908116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.909121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.909567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.910063 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] PATH_INFO: `/v3/services/image` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.910871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.913193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:56.928859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:56.931169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:56.933304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 523/1040] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 31 00:16:56 2022] GET /identity/v3/services/image => generated 86 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:56.934988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.033438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:57.045157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 518/1041] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:16:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:16:57.046567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.078212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044608f-a05a-4826-be71-4148c2cfdada None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.122337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.123245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.123777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.124218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.126047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.128633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.131301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 524/1042] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:57.132642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.172063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:57.177431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167411e8-ebe6-465c-a4db-cd6524a568a2 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.234552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.235422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.235873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.236282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.237242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] RBAC: Adding query filter params (name=image) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:57.237703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.239817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.249046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 519/1043] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:57.250274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.250495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b9028-46b9-494a-9511-492bc3570794 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.302766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.303397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.303667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.303931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.316913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:57.320244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.321748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:16:57.339112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:57.340922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:16:57.342672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:16:57.342672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:16:57.342672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:57.342672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 525/1044] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:16:57.342672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.382835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29427-9809-478c-acdb-82ae7a47ea36 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.435872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.436656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.436980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.437282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.438146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.440617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.472300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784db252-c75c-4592-abe2-629d7e04a8fd None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.521893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.522588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.522862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.523134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.523771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] RBAC: Adding query filter params (type=image) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:57.524071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.525501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.532087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 526/1045] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:16:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:57.533513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.568462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 520/1046] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:16:57 2022] POST /identity/v3/users => generated 285 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:57.569409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.598781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:57.624225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:57.691993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fe47b-f9e5-4cdd-9922-8ab2955498b6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.712627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9cf75-96c2-4413-ad1d-c86941ca5ab0 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.737053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.738203 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.739683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.739683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.739683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:57.740044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.741518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.761226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 527/1047] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:57.762167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.763114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.763597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.764097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:57.765449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:57.767542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:57.767848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:57.852651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 521/1048] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:57.926042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523105e4-fd1e-4053-88a0-df16bb1407b5 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:57.950296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:57.996832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:57.997641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:57.997641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:57.998852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:58.014844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:58.016355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:16:58.034320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:16:58.036167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:16:58.038093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 528/1049] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:16:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:16:58.044653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:58.174528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:58.230078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0c280-9f60-4178-96e4-2cdd58d10e47 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:58.295983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:58.297000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:58.297418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:58.297875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:58.299049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:58.299358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:58.300887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f518bd-2495-4f81-9a7d-81d951c88f93 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:58.318021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 522/1050] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:58.324987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:58.471036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee7e74-c6ef-4772-907b-7a24781405a1 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:58.536557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:58.538067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:58.538835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:58.539635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:58.542577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] RBAC: Adding query filter params (role.id=b4e1d70932c5441e8cbecc0f7b1d41ad, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=f1427404350643b2892469b0da65b788) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:58.543436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:58.546568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1003180-5b34-423a-a73d-d0586774094a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:58.557723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 529/1051] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:16:58 2022] GET /identity/v3/role_assignments?user.id=f1427404350643b2892469b0da65b788&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=b4e1d70932c5441e8cbecc0f7b1d41ad => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:16:59.729336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c40d-e03f-4a53-bebc-4b0e35a18762 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:59.729988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c40d-e03f-4a53-bebc-4b0e35a18762 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:59.729988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c40d-e03f-4a53-bebc-4b0e35a18762 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:59.731244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 523/1052] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:59.737262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70d4e6-a487-412e-8209-337e217a599c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:59.737769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70d4e6-a487-412e-8209-337e217a599c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:59.737769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70d4e6-a487-412e-8209-337e217a599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:59.758823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2d70d4e6-a487-412e-8209-337e217a599c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:59.761077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2d70d4e6-a487-412e-8209-337e217a599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:59.807698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:59.871680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 530/1053] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:16:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:00.145279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c40d-e03f-4a53-bebc-4b0e35a18762 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:00.245038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569326c-4a4d-4320-afd6-710d1c8ed4c9 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.245429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569326c-4a4d-4320-afd6-710d1c8ed4c9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.245699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569326c-4a4d-4320-afd6-710d1c8ed4c9 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.247086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 531/1054] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:00.252838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.253193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.253597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.278346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:00.279556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:00.285531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c40d-e03f-4a53-bebc-4b0e35a18762 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:00.331885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:00.333052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.333703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.334451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.337938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:00.352518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:00.359040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:00.360038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:00.364619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 524/1055] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:00.367681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:00.400599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 532/1056] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:00.502028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0e93d-6084-4257-80be-c774d2abccfa None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:00.547593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:00.548211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.548522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.548797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.549536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:00.549772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:00.551187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02c37e-20e6-4038-b111-6622ba52556a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:00.566063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 525/1057] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:00.567805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:00.655780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03beea-bc06-4174-a2a6-66751b16e59e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.656039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03beea-bc06-4174-a2a6-66751b16e59e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.656249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03beea-bc06-4174-a2a6-66751b16e59e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.660368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 526/1058] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:00.663012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.663313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.663584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.678603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:00.684561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:00.685778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:00.686025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df75de-c30f-41cf-af96-9abe1d48cf71 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:00.727212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:00.728005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.728428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.728796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] PATH_INFO: `/v3/users/35185cdbd7b54b2b85743e53ca38df1e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.743374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] RBAC: Authorizing `identity:get_user(user_id=35185cdbd7b54b2b85743e53ca38df1e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:00.745593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:00.746267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:00.747308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 533/1059] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:00 2022] GET /identity/v3/users/35185cdbd7b54b2b85743e53ca38df1e => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:00.748078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:00.766994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 527/1060] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:00.767836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:00.876044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ba2e-b91d-4235-a515-663b42aafa39 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:00.879783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273324-278b-4810-bfd0-84deeeffd8e8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:00.917100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:00.917620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.917897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.918209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.919474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:00.920150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:00.920459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:00.920739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:00.949806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:00.995414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:01.011446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.012925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.015843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.017448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.135893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 528/1061] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:01.136815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.141589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 534/1062] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:01.143835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.216709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:01.245009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:01.263325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188e6a-11c0-42aa-a397-55abd06ff160 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.267383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f6fb2-cfdc-4902-8e13-4db83316506b None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.302152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.302463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.302787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.303069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.303167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.303343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.303449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.303629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.304872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] RBAC: Authorizing `identity:get_project(project_id=29d2287ff4554f518dbaa0da5f54e637)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.307610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.308796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 535/1063] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:01.309440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.319672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:01.326323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:01.327304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:17:01.328169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:01.332031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 529/1064] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:01.332756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.424104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3a1d-b2a7-4c65-912d-d06d7921dd8c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.443475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc69c3f-f5bf-4ede-a432-dab46084d2e5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.461292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:01.481805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.482772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.483254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.483746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.493077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.496506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.497111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.497381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.498129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=29d2287ff4554f518dbaa0da5f54e637, user.id=35185cdbd7b54b2b85743e53ca38df1e) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:01.498452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.499803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.499957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.501594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.506621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 530/1065] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/role_assignments?user.id=35185cdbd7b54b2b85743e53ca38df1e&scope.project.id=29d2287ff4554f518dbaa0da5f54e637&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:01.508062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:17:01.521663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:01.522804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:17:01.529573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:17:01.529573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:01.529573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:01.529573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 536/1066] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:01.530966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.533798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:01.651247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeb4b1-889b-46e3-8122-449ee068898d None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.694360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bafd3-c20a-40e2-976d-f60e6028ee38 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.700656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.701273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.701549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.701817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.702475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:01.702777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.704301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.714276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 531/1067] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:01.721924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.740028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.740682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.740962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.741231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.741875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:01.742175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.744146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.757579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 537/1068] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:01.817798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:01.857678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03ffe5-339a-46e5-bb2a-26c2b175abe1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:01.899155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:01.899859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:01.900795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:01.901076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:01.915083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:01.917211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:17:01.934828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:01.935846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:17:01.936753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:17:01.936753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:01.936753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:01.936753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 532/1069] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:17:01.943221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:01.995002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:02.091728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e830e-83be-496f-b4e9-e979296b5cfd None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:02.136485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:02.137094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:02.137366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:02.137641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:02.138281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:02.138574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:02.140034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:02.152482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 538/1070] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:02.158465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:02.273490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd9635-7b59-4482-a6f0-e34cc3f9042c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:02.298642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:02.309569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:02.310224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:02.310493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:02.310759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:02.318634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:02.320469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:17:02.333145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:02.336768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:17:02.339418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:02.343479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 533/1071] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:17:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:02.343609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:02.456790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:02.464959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc7665-174a-4498-8f0f-852b968c0c51 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:02.505502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:02.506113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:02.506393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:02.506693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:02.507447 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:02.507807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:02.509252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:02.527681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 539/1072] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:17:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:02.533416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:02.641030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b0019-f6bd-4871-a051-94e24a245474 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:02.675260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:02.675889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:02.676170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:02.676473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:02.691369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:02.693039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:17:02.707535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:02.708619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:17:02.709518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:17:02.709518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:02.709518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:02.709518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 534/1073] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:02.715344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:02.824783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e772a-704e-4b85-9bf6-6be2b954342c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:02.861850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:02.862481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:02.862757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:02.863027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:02.863665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:02.863972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:02.865376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:02.877645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 540/1074] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:02.884066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:02.922084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:03.009594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51687a6b-e239-407f-a26e-f4037f38e9d9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:03.048202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:03.048862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:03.049140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:03.049408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:03.056892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:03.058347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:17:03.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:03.073358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:03.074447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 535/1075] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:17:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:03.079652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:03.100531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:03.208256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5d679-5c47-459a-914b-479afe8b7820 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:03.267864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:03.268508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:03.268787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:03.269089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:03.269918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:03.270241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:03.271674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11edc06-e9ee-4a44-9771-f8f0ed08b653 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:03.285328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 541/1076] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:17:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:03.291898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:03.404267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:03.410567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6071c-4f3b-45fd-be7f-99e9a46723dc None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:03.452301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:03.453036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:03.453255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:03.453522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/f1427404350643b2892469b0da65b788/roles/b4e1d70932c5441e8cbecc0f7b1d41ad` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:03.455774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=f1427404350643b2892469b0da65b788, role_id=b4e1d70932c5441e8cbecc0f7b1d41ad)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:03.457249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:03.501896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f452a44-2913-45ea-9199-c677a524c533 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:03.505157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f452a44-2913-45ea-9199-c677a524c533 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:03.505157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f452a44-2913-45ea-9199-c677a524c533 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:03.505157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 542/1077] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:17:03.509175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:03.509618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:03.510105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:03.529513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:03.530706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:03.546446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 536/1078] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:17:03 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/f1427404350643b2892469b0da65b788/roles/b4e1d70932c5441e8cbecc0f7b1d41ad => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:17:03.613483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 543/1079] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:17:03.886627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:04.018789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca00bee-ce7e-4ae3-87fe-363b8919695e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:04.059241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:04.059885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:04.060166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:04.060481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:04.072121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:04.077012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:04.078507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:04.080298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:04.083271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 537/1080] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:04.084401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:04.087087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:04.193675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ae5d-2c5b-4c69-8d05-bdae17854abb None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:04.231239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:04.231879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:04.232128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:04.232428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:04.233094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:04.233397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:04.234772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:04.243516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 544/1081] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:04.249885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:04.269491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:04.359313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0b72-51d0-4d38-a190-cb98e525edaf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:04.394291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:04.394984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:04.395278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:04.395528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] PATH_INFO: `/v3/users/35185cdbd7b54b2b85743e53ca38df1e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:04.396720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] RBAC: Authorizing `identity:get_user(user_id=35185cdbd7b54b2b85743e53ca38df1e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:04.398163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:04.406254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 538/1082] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:04 2022] GET /identity/v3/users/35185cdbd7b54b2b85743e53ca38df1e => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:04.412160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:04.498732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:04.545745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da78a7-96fe-48c4-a549-cb69afd2ac71 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:04.588931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf2636-9b24-4d5d-9974-37eb36fbdbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:04.589572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf2636-9b24-4d5d-9974-37eb36fbdbf9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:04.589854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf2636-9b24-4d5d-9974-37eb36fbdbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:04.590125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf2636-9b24-4d5d-9974-37eb36fbdbf9 None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:04.591319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf2636-9b24-4d5d-9974-37eb36fbdbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=29d2287ff4554f518dbaa0da5f54e637)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:04.592819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf2636-9b24-4d5d-9974-37eb36fbdbf9 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:04.594537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 545/1083] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:04 2022] GET /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:04.603591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:04.698466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:04.715441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bca16-6337-4555-9417-693a0e95a8ed None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:04.750302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d6e4c-2a2f-42be-8efe-afc134d033d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:04.750957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d6e4c-2a2f-42be-8efe-afc134d033d0 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:04.751233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d6e4c-2a2f-42be-8efe-afc134d033d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:04.751569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d6e4c-2a2f-42be-8efe-afc134d033d0 None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637/users/35185cdbd7b54b2b85743e53ca38df1e/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:04.753456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d6e4c-2a2f-42be-8efe-afc134d033d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=29d2287ff4554f518dbaa0da5f54e637, user_id=35185cdbd7b54b2b85743e53ca38df1e, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:04.754879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d6e4c-2a2f-42be-8efe-afc134d033d0 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:04.855828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 539/1084] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:17:04 2022] PUT /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637/users/35185cdbd7b54b2b85743e53ca38df1e/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:17:05.512422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe9c59-7199-444b-a4d4-52629c83f027 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:05.512976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe9c59-7199-444b-a4d4-52629c83f027 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:05.512976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe9c59-7199-444b-a4d4-52629c83f027 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:05.514435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 546/1085] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:05.519876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:05.520359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:05.520478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:05.539247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:05.541234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:05.613449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 540/1086] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:05.889517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe9c59-7199-444b-a4d4-52629c83f027 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:06.007593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe9c59-7199-444b-a4d4-52629c83f027 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:06.041740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:06.047132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:06.047763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.048038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.048307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.059826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:06.064985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:06.065947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:06.066780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 547/1087] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 00:17:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:06.073462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:06.164973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:06.195254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666cd3a-3101-4fb5-a35f-13f053ab62e2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:06.242532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:06.243191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.244066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.244578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.245439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:06.245745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:06.250080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:06.262884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 541/1088] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:17:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:06.270843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:06.382694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d21f-9212-48e0-a3ce-560f6239477a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:06.422919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:06.423800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.424200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.424514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.438838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:06.440602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:17:06.455917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:06.457470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:17:06.458536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:17:06.458536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:06.458536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:06.458536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 548/1089] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:06.464171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:06.588693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fa5e4-c9ce-4057-8431-6cb566081dcd None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:06.626067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:06.626704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.626969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.627239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.627885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:06.628233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:06.629894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03d22c-4e17-4511-ab20-a46a5810d9a8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:06.632430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:06.644118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 542/1090] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:06.650581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:06.714446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:06.776689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43cf17-a5f1-4dc2-9677-8fd9598bfb90 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:06.818743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:06.819391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.819669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.819938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.823785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856745e3-ecb1-410e-a15a-7f2fbd273c09 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.824169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856745e3-ecb1-410e-a15a-7f2fbd273c09 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.824545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856745e3-ecb1-410e-a15a-7f2fbd273c09 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.826009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 543/1091] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:17:06.830391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:06.831480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:06.831769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:06.831865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:06.832046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:17:06.844989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:06.846100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:06.847181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 549/1092] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 00:17:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:17:06.851537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:06.852036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:06.856767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:06.958387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 544/1093] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:17:06.983509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f84cc-4a8b-45f5-a134-0e60848d1b05 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:06.985412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:07.019925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.020575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.020857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.021123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.021859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:07.022170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:07.023646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:07.049004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 550/1094] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 00:17:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:07.056543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.190298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c389a57-937b-4c64-a3db-6206618f78a9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:07.217881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:07.226823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.230508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.230508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.230508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.268030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.314309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:07.316017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:07.398228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9549-34da-4ca2-80e0-dbc41a5ba4af None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:07.437402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 545/1095] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:07.444048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.450093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.451135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.451645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.452147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.464160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:07.469080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:07.470482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:07.471611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:07.475291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 551/1096] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:17:07.476130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.479682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:07.514518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:07.574421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa34ff-7873-41bf-b319-b4fd524425bf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:07.611758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93178f-d4d4-44ad-bb6e-59db1e67d6d4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:07.626314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.627057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.627262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.627564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.645020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:07.649989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:07.662014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.663799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.663799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.663799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.663799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:07.664537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:07.666283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:17:07.668622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:07.670389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:17:07.672153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:17:07.672153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:17:07.672153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:07.672153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 546/1097] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:17:07.675593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.679452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 552/1098] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:07.687906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.781941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:07.788002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:07.800831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98565-2150-4c03-90b2-61fb68c1aa2f None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:07.819272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a480-1162-4cda-9183-e78d55bf0c42 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:07.839577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.840138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.840457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.840724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] PATH_INFO: `/v3/users/35185cdbd7b54b2b85743e53ca38df1e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.841944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] RBAC: Authorizing `identity:get_user(user_id=35185cdbd7b54b2b85743e53ca38df1e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:07.843423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:07.845292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 553/1099] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/users/35185cdbd7b54b2b85743e53ca38df1e => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:07.852068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:07.861382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:07.862002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:07.862276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:07.862686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:07.863251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:07.863554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:07.864989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:07.880672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 547/1100] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:07.888089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:08.007794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e024d-0400-4b1c-810b-4a587308e0cd None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:08.013326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e279-5af2-4299-9cac-a7e8fa6236b9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:08.051627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:08.052233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:08.052399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:08.052535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:08.052801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:08.052929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:08.053178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:08.053447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:08.063101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:08.064849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:08.066694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:08.076936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:17:08.085103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:08.086187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 548/1101] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:17:08.087647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:08.147345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:08.151668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:08.236907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bb832-44c7-4e4f-8c42-3bf8bf125399 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:08.280739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:08.281352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:08.281626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:08.281898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:08.282588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:08.282886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:08.284351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:08.291571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 554/1102] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:17:08.302207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 549/1103] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:08.302400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:08.303331 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:08.370525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:08.382158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:08.429513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298ba0f-c23a-4f0b-94ce-bf0955e356ca None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:08.432461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93300f-7962-475f-8d8a-f1374df82eba None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:08.469860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:08.470421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:08.470651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:08.470985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:08.471077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:08.471298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:08.471407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:08.471569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:08.472338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] RBAC: Adding query filter params (role.id=b4e1d70932c5441e8cbecc0f7b1d41ad, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=f1427404350643b2892469b0da65b788) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:08.472652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:08.472751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] RBAC: Authorizing `identity:get_project(project_id=29d2287ff4554f518dbaa0da5f54e637)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:08.474094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac682a33-1924-4130-a4f6-62f53c0a01a7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:08.474269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:08.475918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 555/1104] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:08 2022] GET /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:08.484239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:08.488750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 550/1105] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:08 2022] GET /identity/v3/role_assignments?user.id=f1427404350643b2892469b0da65b788&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=b4e1d70932c5441e8cbecc0f7b1d41ad => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:08.612377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd532e1-3d38-44e0-a163-413077a76b23 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:08.636838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:08.666391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:08.667061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:08.667288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:08.667555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:08.668367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=29d2287ff4554f518dbaa0da5f54e637, user.id=35185cdbd7b54b2b85743e53ca38df1e) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:08.668690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:08.670512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1893d8-992f-4910-9d2e-44c8af0db1b6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:08.678382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 556/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:08 2022] GET /identity/v3/role_assignments?user.id=35185cdbd7b54b2b85743e53ca38df1e&scope.project.id=29d2287ff4554f518dbaa0da5f54e637&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:10.714916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e0e98-ee7e-49ff-8677-ba7821c31fb9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:10.715472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e0e98-ee7e-49ff-8677-ba7821c31fb9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:10.715472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e0e98-ee7e-49ff-8677-ba7821c31fb9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:10.717072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 551/1107] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:10.726898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:10.727428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:10.727600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:10.746203 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:10.747411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:10.826210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 557/1108] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:11.098940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e0e98-ee7e-49ff-8677-ba7821c31fb9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:11.230597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e0e98-ee7e-49ff-8677-ba7821c31fb9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:11.257293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:11.270688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c1331-be91-4f45-a681-e49d3969a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:11.271430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c1331-be91-4f45-a681-e49d3969a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:11.271707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c1331-be91-4f45-a681-e49d3969a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:11.274670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c1331-be91-4f45-a681-e49d3969a3c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:11.274670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c1331-be91-4f45-a681-e49d3969a3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:11.275674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c1331-be91-4f45-a681-e49d3969a3c9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:11.278387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 552/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:11.286042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:11.403256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b7250-b736-40cf-b5c7-4eaf877c54e8 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:11.419611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:11.442664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5aed25-8408-4dd0-a1cf-ad2a1489a215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:11.443326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aed25-8408-4dd0-a1cf-ad2a1489a215 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:11.443643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aed25-8408-4dd0-a1cf-ad2a1489a215 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:11.443956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aed25-8408-4dd0-a1cf-ad2a1489a215 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:11.444709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5aed25-8408-4dd0-a1cf-ad2a1489a215 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:11.446268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5aed25-8408-4dd0-a1cf-ad2a1489a215 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:11.630265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 558/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:17:11 2022] POST /identity/v3/users => generated 286 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:17:13.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0383-b32a-42ba-8bc9-40ce285322a7 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:13.648003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0383-b32a-42ba-8bc9-40ce285322a7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:13.648003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d0383-b32a-42ba-8bc9-40ce285322a7 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:13.649401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 553/1111] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:13.655210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:13.655727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:13.655727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:13.675413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:13.676640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:13.747045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 559/1112] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:14.025359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0383-b32a-42ba-8bc9-40ce285322a7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:14.131265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d0383-b32a-42ba-8bc9-40ce285322a7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:14.168688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:14.169323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:14.169623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:14.169894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] PATH_INFO: `/v3/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:14.171328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] RBAC: Authorizing `identity:get_role(role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:14.172758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:14.174106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 554/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:14 2022] GET /identity/v3/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:14.180096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:14.283365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:14.287521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f3f03-f4e9-4dec-8b7a-6bd93ad93dce None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:14.337599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:14.338372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:14.338661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:14.338934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] PATH_INFO: `/v3/users/bd893d8ba7ba42269a718344804e65f0` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:14.353760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] RBAC: Authorizing `identity:get_user(user_id=bd893d8ba7ba42269a718344804e65f0)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:14.355221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:14.356927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 560/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:14 2022] GET /identity/v3/users/bd893d8ba7ba42269a718344804e65f0 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:14.363406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:14.431256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:14.473897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575e185-ad9c-45ee-89fa-e5e15cd5831e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:14.509730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:14.510336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:14.510632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:14.510897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:14.583316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:14.584728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:14.671433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:14.723424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 555/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:14.730196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:14.834611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11041a-5da7-4dd1-9603-bfb9e63dc10d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:14.870839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece188f4-2669-4944-b185-4523d04726db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:14.871471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece188f4-2669-4944-b185-4523d04726db None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:14.871746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece188f4-2669-4944-b185-4523d04726db None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:14.872016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece188f4-2669-4944-b185-4523d04726db None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:14.873174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece188f4-2669-4944-b185-4523d04726db None admin] RBAC: Authorizing `identity:get_project(project_id=29d2287ff4554f518dbaa0da5f54e637)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:14.874657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece188f4-2669-4944-b185-4523d04726db None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:14.876313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 561/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:14 2022] GET /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:14.882571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:14.990564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311e648-6b97-4f30-8149-1d28c2880f6a None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:15.028421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:15.029027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:15.029302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:15.029566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:15.030285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] RBAC: Adding query filter params (role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=29d2287ff4554f518dbaa0da5f54e637, user.id=bd893d8ba7ba42269a718344804e65f0) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:15.030589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:15.032060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde537-81cb-4aef-bc12-5c6a87be972b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:15.037586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 556/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:14 2022] GET /identity/v3/role_assignments?user.id=bd893d8ba7ba42269a718344804e65f0&scope.project.id=29d2287ff4554f518dbaa0da5f54e637&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:16.986251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361f331-ff50-41ee-92d8-2ade2d0355d5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:16.986819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361f331-ff50-41ee-92d8-2ade2d0355d5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:16.986819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361f331-ff50-41ee-92d8-2ade2d0355d5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:16.988123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 562/1118] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:16.993635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:16.994085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:16.994193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:17.014812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:17.016024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:17.034086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:17.089955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 557/1119] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:17.366781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361f331-ff50-41ee-92d8-2ade2d0355d5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:17.407817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:17.488878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361f331-ff50-41ee-92d8-2ade2d0355d5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:17.525997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:17.526687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:17.526902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:17.527172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] PATH_INFO: `/v3/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:17.528631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] RBAC: Authorizing `identity:get_role(role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:17.530011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:17.531347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 563/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:17 2022] GET /identity/v3/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:17.537522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:17.655394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4deb68-fb9c-4cfb-9c65-b3d0b1d3ba1d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:17.690910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:17.691542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:17.691823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:17.692096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] PATH_INFO: `/v3/users/bd893d8ba7ba42269a718344804e65f0` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:17.693282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] RBAC: Authorizing `identity:get_user(user_id=bd893d8ba7ba42269a718344804e65f0)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:17.694783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:17.697024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 558/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:17 2022] GET /identity/v3/users/bd893d8ba7ba42269a718344804e65f0 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:17.703197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:17.814719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:17.821845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7ad68-2aaa-4890-a57f-666452ec195a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:17.857820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c945fa8-9291-4da8-9b45-f5d8f8e7a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:17.858454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c945fa8-9291-4da8-9b45-f5d8f8e7a8d7 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:17.858730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c945fa8-9291-4da8-9b45-f5d8f8e7a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:17.859005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c945fa8-9291-4da8-9b45-f5d8f8e7a8d7 None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:17.860154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c945fa8-9291-4da8-9b45-f5d8f8e7a8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=29d2287ff4554f518dbaa0da5f54e637)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:17.861635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c945fa8-9291-4da8-9b45-f5d8f8e7a8d7 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:17.863351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 564/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:17 2022] GET /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:17.870240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:17.899424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:17.986457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f1db7-ff80-4134-b7be-8e615f95eb9b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:18.023893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e64db6-9849-48d5-8f94-3760aeb06d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:18.025288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e64db6-9849-48d5-8f94-3760aeb06d46 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:18.025565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e64db6-9849-48d5-8f94-3760aeb06d46 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:18.025836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e64db6-9849-48d5-8f94-3760aeb06d46 None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637/users/bd893d8ba7ba42269a718344804e65f0/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:18.028039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e64db6-9849-48d5-8f94-3760aeb06d46 None admin] RBAC: Authorizing `identity:create_grant(project_id=29d2287ff4554f518dbaa0da5f54e637, user_id=bd893d8ba7ba42269a718344804e65f0, role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:18.029635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e64db6-9849-48d5-8f94-3760aeb06d46 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:18.110630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 559/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:17:17 2022] PUT /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637/users/bd893d8ba7ba42269a718344804e65f0/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:17:20.063009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ce89-de41-46b2-9720-04b6f3ce8a81 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:20.063541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ce89-de41-46b2-9720-04b6f3ce8a81 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:20.063541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ce89-de41-46b2-9720-04b6f3ce8a81 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:20.065198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 565/1124] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:20.071108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:20.071556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:20.071673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:20.091993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:20.093253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:20.176458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 560/1125] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:20.450003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ce89-de41-46b2-9720-04b6f3ce8a81 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:20.559350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ce89-de41-46b2-9720-04b6f3ce8a81 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:20.595193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:20.595830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:20.596112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:20.596415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] PATH_INFO: `/v3/roles/0fd816e9ea5448e299d15d32d78ff899` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:20.597852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] RBAC: Authorizing `identity:get_role(role_id=0fd816e9ea5448e299d15d32d78ff899)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:20.599218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:20.600586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 566/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:20 2022] GET /identity/v3/roles/0fd816e9ea5448e299d15d32d78ff899 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:20.607141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:20.621865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:20.712159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5093-8b1c-4e0a-ada0-f0782c6e3116 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:20.754847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:20.755484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:20.755762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:20.756028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] PATH_INFO: `/v3/users/bd893d8ba7ba42269a718344804e65f0` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:20.757196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] RBAC: Authorizing `identity:get_user(user_id=bd893d8ba7ba42269a718344804e65f0)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:20.759431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:20.761156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 561/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:20 2022] GET /identity/v3/users/bd893d8ba7ba42269a718344804e65f0 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:20.767033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:20.798411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:20.870967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33653-01e3-4f11-99d3-42f7477096a0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:20.906835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:20.907476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:20.907768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:20.908058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:20.977047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:20.978530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:21.007258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:21.080021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 567/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:21.086263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:21.166878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:21.188402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25836f7a-d32d-44eb-a733-a4b18ab00943 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:21.224559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300109e-988d-434c-985a-c2f54053d6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:21.225189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300109e-988d-434c-985a-c2f54053d6e1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:21.225464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300109e-988d-434c-985a-c2f54053d6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:21.225734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300109e-988d-434c-985a-c2f54053d6e1 None admin] PATH_INFO: `/v3/projects/29d2287ff4554f518dbaa0da5f54e637` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:21.226947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300109e-988d-434c-985a-c2f54053d6e1 None admin] RBAC: Authorizing `identity:get_project(project_id=29d2287ff4554f518dbaa0da5f54e637)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:21.229162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300109e-988d-434c-985a-c2f54053d6e1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:21.230750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 562/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:21 2022] GET /identity/v3/projects/29d2287ff4554f518dbaa0da5f54e637 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:21.237033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:21.337187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62518fd8-169a-484a-b9cc-3f76dedb8d40 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:21.365413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:21.371200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:21.371859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:21.372132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:21.383437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:21.383437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] RBAC: Adding query filter params (role.id=0fd816e9ea5448e299d15d32d78ff899, scope.project.id=29d2287ff4554f518dbaa0da5f54e637, user.id=bd893d8ba7ba42269a718344804e65f0) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:21.383437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:21.383437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa126-90dc-44e3-af68-bb29f1170a64 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:21.383437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 568/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:21 2022] GET /identity/v3/role_assignments?user.id=bd893d8ba7ba42269a718344804e65f0&scope.project.id=29d2287ff4554f518dbaa0da5f54e637&role.id=0fd816e9ea5448e299d15d32d78ff899 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:23.356661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268df46-6405-419f-b4a3-e177d0b416e1 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:23.357197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268df46-6405-419f-b4a3-e177d0b416e1 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:23.357197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268df46-6405-419f-b4a3-e177d0b416e1 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:23.358549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 563/1131] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:23.364133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:23.364582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:23.364703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:23.383713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:23.385033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:23.455419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 569/1132] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:23.729878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268df46-6405-419f-b4a3-e177d0b416e1 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:23.835477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268df46-6405-419f-b4a3-e177d0b416e1 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:23.871727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e58c51-30b8-4667-9d3d-5330a67e94d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:23.872366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58c51-30b8-4667-9d3d-5330a67e94d1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:23.872648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58c51-30b8-4667-9d3d-5330a67e94d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:23.872914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e58c51-30b8-4667-9d3d-5330a67e94d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:23.874029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e58c51-30b8-4667-9d3d-5330a67e94d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:23.875461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e58c51-30b8-4667-9d3d-5330a67e94d1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:23.877070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 564/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:23.883507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:23.985230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df7ec3-8317-450b-84a5-0e373b81eaa7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:24.009942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:24.020609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5810888-f682-4fb4-bb30-7baf19b91ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:24.022247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810888-f682-4fb4-bb30-7baf19b91ac8 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:24.022247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810888-f682-4fb4-bb30-7baf19b91ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:24.022247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5810888-f682-4fb4-bb30-7baf19b91ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:24.022501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5810888-f682-4fb4-bb30-7baf19b91ac8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:24.024531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5810888-f682-4fb4-bb30-7baf19b91ac8 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:24.170385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 570/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:17:23 2022] POST /identity/v3/projects => generated 311 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:17:26.193960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdca70-a4dc-4887-b944-b9033b328ac0 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:26.194495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdca70-a4dc-4887-b944-b9033b328ac0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:26.194495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdca70-a4dc-4887-b944-b9033b328ac0 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:26.195883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 565/1135] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:17:26.202007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:26.203848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:26.203848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:26.222866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:26.224071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:26.294536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 571/1136] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:17:26.570925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdca70-a4dc-4887-b944-b9033b328ac0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:26.590567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:26.694320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdca70-a4dc-4887-b944-b9033b328ac0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:26.731174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b3906-178e-47e1-ae63-6c81c5ab9fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:26.731811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b3906-178e-47e1-ae63-6c81c5ab9fb5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:26.732091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b3906-178e-47e1-ae63-6c81c5ab9fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:26.732395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b3906-178e-47e1-ae63-6c81c5ab9fb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:26.733649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b3906-178e-47e1-ae63-6c81c5ab9fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:26.735096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b3906-178e-47e1-ae63-6c81c5ab9fb5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:26.736755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 566/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:17:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:26.743315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:26.860148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557233b8-fd7d-47f2-a0f9-f411abdf4d37 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:26.868307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:26.895578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eec94c-02cf-4f94-8bc2-d1f7ee5c7889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:26.896225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eec94c-02cf-4f94-8bc2-d1f7ee5c7889 None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:26.896538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eec94c-02cf-4f94-8bc2-d1f7ee5c7889 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:26.896809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eec94c-02cf-4f94-8bc2-d1f7ee5c7889 None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:26.899234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eec94c-02cf-4f94-8bc2-d1f7ee5c7889 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:26.900032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eec94c-02cf-4f94-8bc2-d1f7ee5c7889 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:27.020291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 572/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:17:26 2022] POST /identity/v3/users => generated 286 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:17:29.098677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367880-2b1c-45cb-a621-7190a29fc82f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:29.099246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367880-2b1c-45cb-a621-7190a29fc82f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:29.099246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367880-2b1c-45cb-a621-7190a29fc82f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:29.101050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 567/1139] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:29.107494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:29.108020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:29.108401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:29.132553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:29.134571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:29.207922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 573/1140] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:29.488964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367880-2b1c-45cb-a621-7190a29fc82f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:29.577706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:29.598513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367880-2b1c-45cb-a621-7190a29fc82f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:29.633884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:29.634752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:29.635040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:29.635381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:29.647141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:29.652561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:29.653726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:29.654629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 568/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:29.659792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:29.760180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:29.768590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3d8a7-caf8-4a28-9704-9d76fc133b40 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:29.802810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:29.803510 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:29.803656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:29.803917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:29.804832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:29.805018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:29.806377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:29.815360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 574/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:29.822647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:29.932630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf4967-b26d-4a88-88ae-cfc40155af8e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:29.970369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:29.971007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:29.971280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:29.971549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] PATH_INFO: `/v3/users/c51e91d4492146698422ada10bb4b4f7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:29.984733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:29.987838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] RBAC: Authorizing `identity:get_user(user_id=c51e91d4492146698422ada10bb4b4f7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:29.989407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:29.991138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 569/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:29 2022] GET /identity/v3/users/c51e91d4492146698422ada10bb4b4f7 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:29.997790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:30.108737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d1da-6e3a-4f6d-9774-ed500b7f71c0 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:30.143797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:30.144424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:30.144697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:30.144961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:30.169374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:30.215568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:30.217130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:30.319195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 575/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:30.325850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:30.427248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b8623-8075-4f85-8b2a-12e1eb2ee327 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:30.463626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfde6c-ece7-4ece-b0bb-cf2fd3c0fa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:30.464253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfde6c-ece7-4ece-b0bb-cf2fd3c0fa19 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:30.464561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfde6c-ece7-4ece-b0bb-cf2fd3c0fa19 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:30.464840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfde6c-ece7-4ece-b0bb-cf2fd3c0fa19 None admin] PATH_INFO: `/v3/projects/0010d9cf94a74bd699757268ea1e19e1` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:30.465965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfde6c-ece7-4ece-b0bb-cf2fd3c0fa19 None admin] RBAC: Authorizing `identity:get_project(project_id=0010d9cf94a74bd699757268ea1e19e1)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:30.467428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfde6c-ece7-4ece-b0bb-cf2fd3c0fa19 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:30.469126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 570/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:30 2022] GET /identity/v3/projects/0010d9cf94a74bd699757268ea1e19e1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:30.475468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:30.532223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:30.577691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce8d24-d84e-4577-b7ec-923f1f170073 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:30.612107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:30.612727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:30.613005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:30.613273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:30.613990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=0010d9cf94a74bd699757268ea1e19e1, user.id=c51e91d4492146698422ada10bb4b4f7) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:30.614293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:30.615744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee92d-42d3-4d48-af7a-67b6d9f44128 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:30.621535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 576/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:30 2022] GET /identity/v3/role_assignments?user.id=c51e91d4492146698422ada10bb4b4f7&scope.project.id=0010d9cf94a74bd699757268ea1e19e1&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:32.537762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad73a6-1be9-46eb-b3d5-97bb77539a4e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:32.538294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad73a6-1be9-46eb-b3d5-97bb77539a4e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:32.538294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad73a6-1be9-46eb-b3d5-97bb77539a4e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:32.539624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 571/1147] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:32.545002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:32.545436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:32.545552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:32.564269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:32.565492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:32.633951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 577/1148] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:32.907685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad73a6-1be9-46eb-b3d5-97bb77539a4e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:32.978712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:33.012536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad73a6-1be9-46eb-b3d5-97bb77539a4e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:33.046569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:33.047179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:33.047451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:33.047717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:33.058579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:33.063336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:33.064718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:33.065863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 572/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:33.070959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:33.119829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:33.172039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88cca-fd22-449e-98fc-2c40bc753f40 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:33.206638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:33.207242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:33.207520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:33.207791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:33.208571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:33.208872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:33.210244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:33.219055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 578/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:33.225176 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:33.325894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57925eb6-72c1-47c6-879c-67b67d7e01d7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:33.352846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:33.360553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:33.361182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:33.365853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:33.365853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] PATH_INFO: `/v3/users/c51e91d4492146698422ada10bb4b4f7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:33.365853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] RBAC: Authorizing `identity:get_user(user_id=c51e91d4492146698422ada10bb4b4f7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:33.365853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:33.366173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 573/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:33 2022] GET /identity/v3/users/c51e91d4492146698422ada10bb4b4f7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:33.372062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:33.471679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fc50e-8674-4682-9112-2a93c191305e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:33.482501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:33.505469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4e52a-3f5c-4f81-9cb0-e460cf569c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:33.506095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4e52a-3f5c-4f81-9cb0-e460cf569c42 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:33.506376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4e52a-3f5c-4f81-9cb0-e460cf569c42 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:33.506649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4e52a-3f5c-4f81-9cb0-e460cf569c42 None admin] PATH_INFO: `/v3/projects/0010d9cf94a74bd699757268ea1e19e1` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:33.508411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4e52a-3f5c-4f81-9cb0-e460cf569c42 None admin] RBAC: Authorizing `identity:get_project(project_id=0010d9cf94a74bd699757268ea1e19e1)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:33.509848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4e52a-3f5c-4f81-9cb0-e460cf569c42 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:33.511490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 579/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:33 2022] GET /identity/v3/projects/0010d9cf94a74bd699757268ea1e19e1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:33.517556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:33.618155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e74dfe-b676-4c36-ac93-433e21e0a2f7 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:33.653024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b751dd1-0133-4e56-ad3e-6cceed4991c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:33.653683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b751dd1-0133-4e56-ad3e-6cceed4991c9 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:33.653966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b751dd1-0133-4e56-ad3e-6cceed4991c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:33.654242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b751dd1-0133-4e56-ad3e-6cceed4991c9 None admin] PATH_INFO: `/v3/projects/0010d9cf94a74bd699757268ea1e19e1/users/c51e91d4492146698422ada10bb4b4f7/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:33.656200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b751dd1-0133-4e56-ad3e-6cceed4991c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0010d9cf94a74bd699757268ea1e19e1, user_id=c51e91d4492146698422ada10bb4b4f7, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:33.657669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b751dd1-0133-4e56-ad3e-6cceed4991c9 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:33.772330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 574/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:17:33 2022] PUT /identity/v3/projects/0010d9cf94a74bd699757268ea1e19e1/users/c51e91d4492146698422ada10bb4b4f7/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:17:35.678113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96fee-1aa3-449f-a7c2-c92c81042590 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:35.678714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96fee-1aa3-449f-a7c2-c92c81042590 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:35.678714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96fee-1aa3-449f-a7c2-c92c81042590 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:35.680058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 580/1154] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:35.686112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:35.686567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:35.686674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:35.705844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:35.707069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:35.759831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:35.776843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 575/1155] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:17:36.048101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96fee-1aa3-449f-a7c2-c92c81042590 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:36.153457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96fee-1aa3-449f-a7c2-c92c81042590 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:36.188197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:36.189123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:36.189402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:36.189674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:36.202100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:36.206930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:36.208116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:36.209052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 581/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:36.214188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:36.312623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ebbfa-efd3-476d-92ff-a4f1f127405b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:36.348165 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:36.348799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:36.349078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:36.349347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:36.350007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:36.350306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:36.351681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:36.359972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 576/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:36.365874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:36.409612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:36.471264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c7c08-5a8f-4e28-9d84-07096a39d39c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:36.507310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:36.507940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:36.508214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:36.508508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] PATH_INFO: `/v3/users/c51e91d4492146698422ada10bb4b4f7` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:36.509618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] RBAC: Authorizing `identity:get_user(user_id=c51e91d4492146698422ada10bb4b4f7)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:36.511046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:36.512991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 582/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:36 2022] GET /identity/v3/users/c51e91d4492146698422ada10bb4b4f7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:36.519003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:36.569912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:36.621441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567f131-d42e-48b5-b15b-417b61431eb8 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:36.654711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:36.655311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:36.655588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:36.655863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:36.723060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:36.724543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:36.772409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:36.824235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 577/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:36.830253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:36.931544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a57890-030c-4a71-93bf-f97aa4382b0d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:36.938912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:36.968007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4513b-1ad2-415c-9385-95256eebd6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:36.968676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4513b-1ad2-415c-9385-95256eebd6c1 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:36.968954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4513b-1ad2-415c-9385-95256eebd6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:36.969224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4513b-1ad2-415c-9385-95256eebd6c1 None admin] PATH_INFO: `/v3/projects/0010d9cf94a74bd699757268ea1e19e1` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:36.970354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4513b-1ad2-415c-9385-95256eebd6c1 None admin] RBAC: Authorizing `identity:get_project(project_id=0010d9cf94a74bd699757268ea1e19e1)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:36.971805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4513b-1ad2-415c-9385-95256eebd6c1 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:36.973514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 583/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:36 2022] GET /identity/v3/projects/0010d9cf94a74bd699757268ea1e19e1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:36.980505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:37.079421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5c514-53da-4050-ae59-fc1360c3ea6f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:37.113189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:37.113777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:37.114050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:37.114316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:37.115015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=0010d9cf94a74bd699757268ea1e19e1, user.id=c51e91d4492146698422ada10bb4b4f7) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:37.115315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:37.116769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe95bbb-d162-4aae-941b-d17eabe2d1d2 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:37.122734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 578/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:36 2022] GET /identity/v3/role_assignments?user.id=c51e91d4492146698422ada10bb4b4f7&scope.project.id=0010d9cf94a74bd699757268ea1e19e1&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:39.065986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201bcb8-6636-4ac0-80a1-b087eeb25d5d None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:39.066539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201bcb8-6636-4ac0-80a1-b087eeb25d5d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:39.066539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201bcb8-6636-4ac0-80a1-b087eeb25d5d None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:39.067839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 584/1162] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:39.073271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:39.074491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:39.074491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:39.093073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:39.094287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:39.102420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:39.173226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 579/1163] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:39.454383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201bcb8-6636-4ac0-80a1-b087eeb25d5d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:39.556645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201bcb8-6636-4ac0-80a1-b087eeb25d5d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:39.591719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:39.592316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:39.592616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:39.592883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:39.636853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:39.671790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:39.673564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:39.776546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 585/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:39.783372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:39.883589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc48dee-9299-4ba9-9d2f-329e5ac8f53f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:39.924443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:39.925098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:39.925371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:39.925639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:39.933576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:39.935148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-59b6aa23-b75c-4160-af31-f86e7db6e086 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:39.946987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:39.948199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:39.949111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 580/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:17:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:39.954404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:40.030293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:40.057349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4027f6b-e209-4d3c-aadd-f1635cad9d6e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:40.094257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:40.094933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:40.095109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:40.095378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:40.096013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:40.096310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:40.097690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ded5c2-29c8-45b6-93a1-c2d591420044 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:40.106292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 586/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 00:17:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:42.017842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b29ad-65de-48dc-a61d-4b4996893b20 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:42.018374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b29ad-65de-48dc-a61d-4b4996893b20 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:42.018374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b29ad-65de-48dc-a61d-4b4996893b20 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:42.019713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 581/1167] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:42.024946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01a005-eb95-4f85-b10e-20073ce52484 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:42.025402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01a005-eb95-4f85-b10e-20073ce52484 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:42.025492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01a005-eb95-4f85-b10e-20073ce52484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:42.045959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2c01a005-eb95-4f85-b10e-20073ce52484 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:42.047156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2c01a005-eb95-4f85-b10e-20073ce52484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:42.115691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 587/1168] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:42.387219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b29ad-65de-48dc-a61d-4b4996893b20 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:42.390071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:42.489056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b29ad-65de-48dc-a61d-4b4996893b20 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:42.522401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8f8db-696a-4e3e-97fa-5a4706af0ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:42.523006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8f8db-696a-4e3e-97fa-5a4706af0ab8 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:42.523284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8f8db-696a-4e3e-97fa-5a4706af0ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:42.523554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8f8db-696a-4e3e-97fa-5a4706af0ab8 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:42.524309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8f8db-696a-4e3e-97fa-5a4706af0ab8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:42.525840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8f8db-696a-4e3e-97fa-5a4706af0ab8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:42.608007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 582/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Wed Aug 31 00:17:42 2022] POST /identity/v3/domains => generated 267 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:17:44.568911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3c4a5-a441-41d7-af60-3fa8a2f2f34b None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:44.569467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3c4a5-a441-41d7-af60-3fa8a2f2f34b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:44.569467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3c4a5-a441-41d7-af60-3fa8a2f2f34b None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:44.570898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 588/1170] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:44.577088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:44.577514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:44.577632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:44.603892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:44.605362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:44.666676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:44.678158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 583/1171] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:44.954561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3c4a5-a441-41d7-af60-3fa8a2f2f34b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:45.045354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:45.086539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3c4a5-a441-41d7-af60-3fa8a2f2f34b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:45.134158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505aaeec-536e-45d2-87d3-ce639cd10a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:45.134828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505aaeec-536e-45d2-87d3-ce639cd10a2a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:45.135095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505aaeec-536e-45d2-87d3-ce639cd10a2a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:45.135363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505aaeec-536e-45d2-87d3-ce639cd10a2a None admin] PATH_INFO: `/v3/domains/98abb39981c34759b9882f588f3e94d1` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:45.147010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505aaeec-536e-45d2-87d3-ce639cd10a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=98abb39981c34759b9882f588f3e94d1)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:45.148614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505aaeec-536e-45d2-87d3-ce639cd10a2a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:45.150355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 589/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 31 00:17:44 2022] GET /identity/v3/domains/98abb39981c34759b9882f588f3e94d1 => generated 267 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:45.157267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:45.275390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf07d-6b5d-4e34-a950-93c08ada9a15 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:45.318806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef753020-6bae-4a73-93fd-a089fb1450bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:45.319793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753020-6bae-4a73-93fd-a089fb1450bb None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:45.320262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753020-6bae-4a73-93fd-a089fb1450bb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:45.320750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef753020-6bae-4a73-93fd-a089fb1450bb None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:45.321864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef753020-6bae-4a73-93fd-a089fb1450bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:45.324724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef753020-6bae-4a73-93fd-a089fb1450bb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:45.423471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 584/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:17:45 2022] POST /identity/v3/projects => generated 361 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:17:47.491424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b69af-cf89-472a-bb2f-ed1ad6d92e46 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:47.491924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b69af-cf89-472a-bb2f-ed1ad6d92e46 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:47.492034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b69af-cf89-472a-bb2f-ed1ad6d92e46 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:47.493332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 590/1174] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:17:47.498853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:47.499274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:47.499407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:47.518207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:47.519406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:47.565778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:47.596100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 585/1175] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:47.869078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b69af-cf89-472a-bb2f-ed1ad6d92e46 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:47.999215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b69af-cf89-472a-bb2f-ed1ad6d92e46 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:48.033242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:48.042770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d210e5-8b1a-4e2d-a770-407818bcb30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:48.043687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d210e5-8b1a-4e2d-a770-407818bcb30b None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:48.044165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d210e5-8b1a-4e2d-a770-407818bcb30b None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:48.044650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d210e5-8b1a-4e2d-a770-407818bcb30b None admin] PATH_INFO: `/v3/domains/98abb39981c34759b9882f588f3e94d1` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:48.050404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d210e5-8b1a-4e2d-a770-407818bcb30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=98abb39981c34759b9882f588f3e94d1)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:48.050404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d210e5-8b1a-4e2d-a770-407818bcb30b None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:48.052303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 591/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 31 00:17:47 2022] GET /identity/v3/domains/98abb39981c34759b9882f588f3e94d1 => generated 267 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:48.061050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:48.176193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718f7f3-f226-4248-ba49-5a7fa54b41eb None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:48.221861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065e10e-e984-4713-9ff3-73cb70fff01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:48.222807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e10e-e984-4713-9ff3-73cb70fff01e None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:48.223239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e10e-e984-4713-9ff3-73cb70fff01e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:48.223662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065e10e-e984-4713-9ff3-73cb70fff01e None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:48.224803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065e10e-e984-4713-9ff3-73cb70fff01e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:48.227286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065e10e-e984-4713-9ff3-73cb70fff01e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:48.357675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 586/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:17:48 2022] POST /identity/v3/users => generated 311 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:17:50.533776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388256-33f1-446f-aef9-d86ad3a21d62 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:50.534428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388256-33f1-446f-aef9-d86ad3a21d62 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:50.534428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388256-33f1-446f-aef9-d86ad3a21d62 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:50.535664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 592/1178] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:17:50.541089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:50.541534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:50.541765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:50.567651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:50.568900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:50.595436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:50.641644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 587/1179] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:50.920834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388256-33f1-446f-aef9-d86ad3a21d62 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:51.033411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388256-33f1-446f-aef9-d86ad3a21d62 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:51.070871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:51.071497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:51.071773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:51.072043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:51.083709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:51.089241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:51.090651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:51.091700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 593/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:51.097287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:51.206049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033320-4fca-489e-bb6b-756f5d106315 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:51.243960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:51.244726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:51.245593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:51.245877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:51.246546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:51.246848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:51.248228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:51.257064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 588/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:51.265026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:51.325221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:51.375154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ed58f-82ef-447e-807e-df5b152bf550 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:51.419876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:51.420544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:51.428778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:51.428778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] PATH_INFO: `/v3/users/69b0137ba10940cd8d8f3b6bef564f50` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:51.436732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] RBAC: Authorizing `identity:get_user(user_id=69b0137ba10940cd8d8f3b6bef564f50)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:51.438424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:51.440096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 594/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:51 2022] GET /identity/v3/users/69b0137ba10940cd8d8f3b6bef564f50 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:51.446214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:51.473824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:51.558129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455f912-f1b3-44b7-a50f-a44c1d4e7d34 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:51.595236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:51.595851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:51.596132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:51.596429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:51.673820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:51.675292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:51.700461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:51.780067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 589/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:17:51.786282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:51.890670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76b3a8-c7d7-42f8-b789-192ac7835676 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:51.904948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:51.924706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0ea5-ab40-4a09-8037-d2d74fd8ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:51.925384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0ea5-ab40-4a09-8037-d2d74fd8ab47 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:51.925658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0ea5-ab40-4a09-8037-d2d74fd8ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:51.925928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0ea5-ab40-4a09-8037-d2d74fd8ab47 None admin] PATH_INFO: `/v3/projects/e4639f55e30049d4a2ede854a7ac28f4` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:51.927043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b0ea5-ab40-4a09-8037-d2d74fd8ab47 None admin] RBAC: Authorizing `identity:get_project(project_id=e4639f55e30049d4a2ede854a7ac28f4)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:51.928507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b0ea5-ab40-4a09-8037-d2d74fd8ab47 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:51.930136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 595/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:51 2022] GET /identity/v3/projects/e4639f55e30049d4a2ede854a7ac28f4 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:51.938642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:52.041386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba13922-06c4-4c4a-9526-533fb4180907 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:52.064796 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:52.077476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:52.080551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:52.080551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:52.080551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:52.080551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=e4639f55e30049d4a2ede854a7ac28f4, user.id=69b0137ba10940cd8d8f3b6bef564f50) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:52.080551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:52.081199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec372-ead1-4387-9ad9-d40ef54a26a1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:52.086725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 590/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:51 2022] GET /identity/v3/role_assignments?user.id=69b0137ba10940cd8d8f3b6bef564f50&scope.project.id=e4639f55e30049d4a2ede854a7ac28f4&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:53.994735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c765485-4c87-4853-945c-0cb549fc6720 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:53.995268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c765485-4c87-4853-945c-0cb549fc6720 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:53.995268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c765485-4c87-4853-945c-0cb549fc6720 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:53.996653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 596/1186] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:54.002126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:54.002583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:54.002677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:54.020824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:54.022004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:54.098390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 591/1187] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:54.369252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c765485-4c87-4853-945c-0cb549fc6720 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:54.473043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c765485-4c87-4853-945c-0cb549fc6720 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:54.507504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:54.508124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:54.508436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:54.508702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:54.519634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:54.524369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:54.528870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:17:54.529928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 597/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:54.531841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:54.632809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5849050-dc3c-4b62-b6fa-2485c5f75c89 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:54.649394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:54.666768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:54.672440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:54.672440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:54.672440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:54.672440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:54.672440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:54.672440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:54.678438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 592/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:54.685057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:54.730900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:54.789643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a3754-a4b1-41e1-9a77-a1530ee7058e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:54.824812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:54.825427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:54.825697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:54.825961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] PATH_INFO: `/v3/users/69b0137ba10940cd8d8f3b6bef564f50` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:54.827054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] RBAC: Authorizing `identity:get_user(user_id=69b0137ba10940cd8d8f3b6bef564f50)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:54.828500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:54.830068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 598/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:54 2022] GET /identity/v3/users/69b0137ba10940cd8d8f3b6bef564f50 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:17:54.836126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:54.936566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd84a18-040b-4c1f-a5fe-f19026f054ab None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:54.970749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b85d37-d178-4cc3-a831-8a55a35e6c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:54.971371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b85d37-d178-4cc3-a831-8a55a35e6c4f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:54.971656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b85d37-d178-4cc3-a831-8a55a35e6c4f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:54.971930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b85d37-d178-4cc3-a831-8a55a35e6c4f None admin] PATH_INFO: `/v3/projects/e4639f55e30049d4a2ede854a7ac28f4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:54.973079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b85d37-d178-4cc3-a831-8a55a35e6c4f None admin] RBAC: Authorizing `identity:get_project(project_id=e4639f55e30049d4a2ede854a7ac28f4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:54.974518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b85d37-d178-4cc3-a831-8a55a35e6c4f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:54.976151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 593/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:54 2022] GET /identity/v3/projects/e4639f55e30049d4a2ede854a7ac28f4 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:54.982425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:55.082551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca35923-e451-4a2c-84a4-9803bfa72d59 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:55.094387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:55.117879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548288-a8b3-481c-9cea-861de9e4ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:55.118587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548288-a8b3-481c-9cea-861de9e4ffa3 None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:55.118842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548288-a8b3-481c-9cea-861de9e4ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:55.119112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548288-a8b3-481c-9cea-861de9e4ffa3 None admin] PATH_INFO: `/v3/projects/e4639f55e30049d4a2ede854a7ac28f4/users/69b0137ba10940cd8d8f3b6bef564f50/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:55.121330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548288-a8b3-481c-9cea-861de9e4ffa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4639f55e30049d4a2ede854a7ac28f4, user_id=69b0137ba10940cd8d8f3b6bef564f50, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:55.122675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548288-a8b3-481c-9cea-861de9e4ffa3 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:55.218766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 599/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:17:54 2022] PUT /identity/v3/projects/e4639f55e30049d4a2ede854a7ac28f4/users/69b0137ba10940cd8d8f3b6bef564f50/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:17:57.128865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1fbcd-b52d-4a5d-9e80-f0c898e6c10f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:57.129413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1fbcd-b52d-4a5d-9e80-f0c898e6c10f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:57.129413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1fbcd-b52d-4a5d-9e80-f0c898e6c10f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:57.130909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 594/1193] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:17:57.136104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:57.136600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:57.136756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:57.155066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:17:57.156264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:17:57.224498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 600/1194] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:17:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:17:57.499416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1fbcd-b52d-4a5d-9e80-f0c898e6c10f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:57.552399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:57.610366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1fbcd-b52d-4a5d-9e80-f0c898e6c10f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:57.646737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:57.647374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:57.647650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:57.647917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:57.659266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:17:57.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:17:57.665338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:17:57.666545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 595/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:17:57.671710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:57.770708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:57.773661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede89a-d30c-4ce1-9af9-0788c083619d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:57.814018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:57.814626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:57.814902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:57.815196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:57.815854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:57.816155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:57.817584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:57.829557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 601/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:17:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:57.835331 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:57.947714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47862-108a-454b-bc47-bc28339e9e69 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:57.954754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:57.982607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:57.983341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:57.983617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:57.983891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] PATH_INFO: `/v3/users/69b0137ba10940cd8d8f3b6bef564f50` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:57.985118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] RBAC: Authorizing `identity:get_user(user_id=69b0137ba10940cd8d8f3b6bef564f50)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:57.986614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:57.988525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 596/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 00:17:57 2022] GET /identity/v3/users/69b0137ba10940cd8d8f3b6bef564f50 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:57.994686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:58.101501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809da1-0872-4ae6-a63f-e557ce5e0538 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:58.139426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:58.140086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:58.140343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:58.140614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:58.172001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:58.210873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:58.212314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:58.313522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 602/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:17:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:17:58.319455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:58.422711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5747b95-6805-44fe-84b3-742b5e9641a1 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:58.456717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0d27-0dfd-414f-aef7-c033d0f268dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:58.457385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0d27-0dfd-414f-aef7-c033d0f268dd None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:58.457689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0d27-0dfd-414f-aef7-c033d0f268dd None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:58.457967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0d27-0dfd-414f-aef7-c033d0f268dd None admin] PATH_INFO: `/v3/projects/e4639f55e30049d4a2ede854a7ac28f4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:58.459083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac0d27-0dfd-414f-aef7-c033d0f268dd None admin] RBAC: Authorizing `identity:get_project(project_id=e4639f55e30049d4a2ede854a7ac28f4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:58.460548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac0d27-0dfd-414f-aef7-c033d0f268dd None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:58.462249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 597/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 00:17:58 2022] GET /identity/v3/projects/e4639f55e30049d4a2ede854a7ac28f4 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:17:58.469251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:17:58.532508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:58.571234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f5d77-0596-4ed9-8cd4-fba379433d07 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:17:58.607970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:17:58.608604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:17:58.608879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:17:58.609149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:17:58.609877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=e4639f55e30049d4a2ede854a7ac28f4, user.id=69b0137ba10940cd8d8f3b6bef564f50) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:17:58.610173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:17:58.611598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbca823-e25b-4737-8784-a82d43740eeb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:17:58.617944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 603/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:17:58 2022] GET /identity/v3/role_assignments?user.id=69b0137ba10940cd8d8f3b6bef564f50&scope.project.id=e4639f55e30049d4a2ede854a7ac28f4&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:18:08.452384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=79 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:18:08.615056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=37 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:14.866071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7828a6-a184-424e-b273-3fa5f70a5ec6 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:14.866672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7828a6-a184-424e-b273-3fa5f70a5ec6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:14.866672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7828a6-a184-424e-b273-3fa5f70a5ec6 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:14.867946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 598/1201] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:14.873278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:14.873966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:14.873966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:14.893125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:14.894319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:14.967364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 604/1202] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:15.241925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7828a6-a184-424e-b273-3fa5f70a5ec6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:15.348181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7828a6-a184-424e-b273-3fa5f70a5ec6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:15.384003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:15.384652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:15.384931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:15.385201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:15.395746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:15.398353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:15.412151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:15.413369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:15.414669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:15.414669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:15.414669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:15.414669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 599/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:15.419440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:15.522049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c413ce5-de11-4ae7-8110-f9d81dca4cf7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:15.552387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:15.558816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:15.559425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:15.559696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:15.560215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:15.560877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:15.561174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:15.562608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116f80b-de13-4fd1-9025-05ab7d444718 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:15.573899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 605/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:15.579941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:15.634969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:15.683591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13e6f-90e3-4194-bd50-577b2dfc32cb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:15.719594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0788c-506d-4a98-8217-9e5053f9eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:15.720198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0788c-506d-4a98-8217-9e5053f9eec1 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:15.720509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0788c-506d-4a98-8217-9e5053f9eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:15.720782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0788c-506d-4a98-8217-9e5053f9eec1 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:15.721417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0788c-506d-4a98-8217-9e5053f9eec1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:15.722856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0788c-506d-4a98-8217-9e5053f9eec1 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:15.839791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 600/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 00:19:15 2022] POST /identity/v3/users => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:19:17.814202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaef91-9a46-4b8b-bb7e-70687b6228c7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:17.814790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaef91-9a46-4b8b-bb7e-70687b6228c7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:17.814790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaef91-9a46-4b8b-bb7e-70687b6228c7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:17.816152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 606/1206] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:17.822164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:17.822628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:17.822729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:17.841610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:17.842827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:17.913746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 601/1207] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:18.187339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaef91-9a46-4b8b-bb7e-70687b6228c7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:18.294311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaef91-9a46-4b8b-bb7e-70687b6228c7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:18.331844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:18.332669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:18.332953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:18.333216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:18.346215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:19:18.351257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:18.352458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:18.353716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 607/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:19:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:18.358673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:18.399529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:18.461352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc5d3c-b34e-4797-841e-22e5f56155ac None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:18.496137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:18.496778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:18.497049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:18.497313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:18.497969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:18.498263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:18.499708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:18.508576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 602/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:19:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:18.514397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:18.566231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:18.616704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464348b-1628-4ae3-b2a0-312c026e99ca None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:18.649400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:18.650020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:18.650300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:18.650563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:18.660950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:18.662418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:18.677744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:18.679618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:18.680715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:18.680715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:18.680715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:18.680715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 608/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:18.685723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:18.790156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:18.793163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6584ceb-f704-4b41-8ada-9b6a7db81e32 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:18.829392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:18.830094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:18.830261 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:18.830532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:18.831172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:18.831475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:18.832881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:18.845205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 603/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:18.852038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:18.956024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16159ff4-cc00-4e4d-b9a2-25ee35bfcdc6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:18.957796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:18.991825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:18.992495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:18.992771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:18.993040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] PATH_INFO: `/v3/users/placement` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:19.001567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:19.003052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:19:19.014887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:19.016134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:19.017201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 609/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:19.022950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:19.126692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b41df-2b83-401d-b7a2-09928e14634c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:19.160996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:19.161590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:19.161870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:19.162138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:19.163007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:19.163313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:19.164776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:19.183332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 604/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 00:19:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:19.189542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:19.297513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0799b-791b-4c32-9e95-706b30b459af None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:19.333389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:19.335415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:19.335415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:19.335415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:19.365505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:19.405413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:19.406817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:19.508545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 610/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:19:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:19:19.514512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:19.533974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:19.619634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c19f0-7c67-429e-9e50-c0b1dece2678 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:19.656127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:19.656704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:19.657001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:19.657275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:19.668109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:19.669619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:19.685434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:19.686729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:19.687749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:19.687749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:19.687749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:19.687749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 605/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:19.692629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:19.754891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:19.794743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3f5ad-0b5a-4f0b-a4b0-a9d59544fdff None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:19.831293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:19.831894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:19.832165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:19.832459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:19.833086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:19.834730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:19.834730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:19.846278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 611/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:19.852561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:19.923608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:19.956717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ea203-b1e6-47ca-9f7a-9f3ea9de396f None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:19.990637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:19.991254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:19.991535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:19.992506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:19.999164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:20.000638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:19:20.012001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:20.013097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:20.014200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 606/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:20.019096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:20.122339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e345bda-d967-4db2-8916-fcc4cdf90cea None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:20.144240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:20.155803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:20.156465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:20.156744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:20.157016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:20.157723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:20.158028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:20.159445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c959a-d885-4925-b2e1-feef86bd12fb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:20.171789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 612/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:19:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:20.178104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:20.279309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c858c77-2cc8-478e-bede-9b9ee9843477 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:20.308592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:20.314544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:20.315149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:20.315544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:20.315691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:20.316546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:20.318934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:20.318934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4c60e-a5ea-4cb1-b18d-923e6a592a58 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:20.324112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 607/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:19:20 2022] GET /identity/v3/role_assignments?user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:22.250232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f511c-cfbc-48be-90f9-fb632aedaf7c None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:22.250858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f511c-cfbc-48be-90f9-fb632aedaf7c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:22.250858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f511c-cfbc-48be-90f9-fb632aedaf7c None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:22.252135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 613/1220] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:22.257823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:22.258269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:22.258482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:22.277295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:22.278503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:22.347834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 608/1221] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:22.625281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f511c-cfbc-48be-90f9-fb632aedaf7c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:22.728607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f511c-cfbc-48be-90f9-fb632aedaf7c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:22.761989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:22.762605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:22.762878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:22.763184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:22.773702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:19:22.778380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:22.779549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:22.780632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 614/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:19:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:22.785647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:22.888310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69542a-539d-4471-9a3c-80573ef66e0a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:22.914635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:22.925637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:22.926325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:22.926614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:22.926899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:22.927554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] RBAC: Adding query filter params (name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:22.927855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:22.929694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:22.938242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 609/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:19:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:22.944353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:22.989256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:23.048858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f23bf-1182-4d1d-a1d0-fdf758c276c4 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:23.083952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:23.084614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:23.084892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:23.085770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:23.096013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:23.097585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:23.113247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:23.114465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:23.115598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:23.115598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:23.115598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:23.115598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 615/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:23.120621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:23.225445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4aac4-58f1-45f1-8d5a-15b64d3f7b2b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:23.259723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:23.260375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:23.260627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:23.260895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:23.261539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:23.261847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:23.263216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:23.275845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 610/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:23.286968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:23.377222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:23.384017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951023c-7116-4c25-9ba8-96ce954bcf68 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:23.418672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:23.419365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:23.419645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:23.419918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:23.427834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:23.429395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:19:23.440691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:23.442116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:23.443259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 616/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:23.447842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:23.496257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:23.548742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a7a19-f141-4bfc-836d-3b92cc0d4e55 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:23.582600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:23.583195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:23.583468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:23.583734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:23.584476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:23.584773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:23.586360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:23.603551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 611/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 00:19:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:23.609291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:23.712779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d35df-3422-45e3-9140-426695d14555 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:23.747675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:23.748296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:23.755483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:23.755483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:23.759305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:23.760838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:23.774968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:23.776163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:23.777476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:23.777476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:23.777476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:23.777476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 617/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:23.782075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:23.886521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce65e79-7d23-4ed9-b10e-1399cc0b9036 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:23.888841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:23.920087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:23.920714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:23.921022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:23.921337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:23.922006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:23.922322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:23.924273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:23.936452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 612/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:23.942568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:23.952556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:24.043296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc9d7-f707-4519-aa0d-b0d58869a435 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:24.078450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77263022-967e-4cee-b669-c24095923c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:24.079065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77263022-967e-4cee-b669-c24095923c70 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:24.079337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77263022-967e-4cee-b669-c24095923c70 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:24.079603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77263022-967e-4cee-b669-c24095923c70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:24.086385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77263022-967e-4cee-b669-c24095923c70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:24.087855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77263022-967e-4cee-b669-c24095923c70 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-77263022-967e-4cee-b669-c24095923c70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:19:24.099239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:24.100167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:24.101248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 618/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:19:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:24.106797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:24.210572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1638d-5a5f-4883-9926-871f5dc118ed None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:24.244100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:24.244734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:24.245015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:24.245287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:24.246044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:24.246346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:24.247778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e6073-d826-4cd8-87c5-e883093a3dd8 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:24.260811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 613/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:24.266951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77263022-967e-4cee-b669-c24095923c70 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:24.335688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:24.371060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77263022-967e-4cee-b669-c24095923c70 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:24.405512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412861b-6867-4711-ba5a-85e7e87fd8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:24.406560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412861b-6867-4711-ba5a-85e7e87fd8aa None admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:24.406892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412861b-6867-4711-ba5a-85e7e87fd8aa None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:24.407211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412861b-6867-4711-ba5a-85e7e87fd8aa None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/42f4dfa35f464a3f9cdbbc1e1ee4a980/roles/deb6dc5f801e411882108de755caff96` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:24.423173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412861b-6867-4711-ba5a-85e7e87fd8aa None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=42f4dfa35f464a3f9cdbbc1e1ee4a980, role_id=deb6dc5f801e411882108de755caff96)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:24.425370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412861b-6867-4711-ba5a-85e7e87fd8aa None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:24.541716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 619/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:19:24 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/42f4dfa35f464a3f9cdbbc1e1ee4a980/roles/deb6dc5f801e411882108de755caff96 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:19:26.463707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4fd20-1011-40a6-9427-491d3bb13e61 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:26.464285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4fd20-1011-40a6-9427-491d3bb13e61 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:26.464285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4fd20-1011-40a6-9427-491d3bb13e61 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:26.465616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 614/1233] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:19:26.470913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:26.471390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:26.471517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:26.490509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:26.492250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:26.560546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 620/1234] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:26.846295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4fd20-1011-40a6-9427-491d3bb13e61 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:26.873112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:26.951141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4fd20-1011-40a6-9427-491d3bb13e61 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:26.986320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:26.986945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:26.987219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:26.987483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:27.002363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:19:27.007257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:27.008526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:27.009566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 615/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 00:19:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:27.014841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:27.063518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:27.115990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d49e74-ee54-4e40-abb8-77ef1c371ce2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:27.152593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:27.153206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:27.153481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:27.153747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:27.154406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] RBAC: Adding query filter params (name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:27.154702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:27.156064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:27.164659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 621/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 00:19:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:19:27.170991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:27.257981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:27.273848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0336d-8066-4a62-97d0-652e037be1ed None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:27.307150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:27.307773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:27.308053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:27.308341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:27.318943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:27.320723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:27.335696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:27.337686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:27.338798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:27.338798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:27.338798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:27.338798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 616/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:27.343126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:27.449434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261a1e-dd95-496b-900d-a12603988892 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:27.458274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:27.483413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:27.484028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:27.484307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:27.484617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:27.485261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:27.485565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:27.486922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:27.498792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 622/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:27.504779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:27.606615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea951b8f-bec9-4867-8317-8aea3cfc2e08 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:27.641551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:27.642201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:27.642641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:27.642909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] PATH_INFO: `/v3/users/placement` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:27.645658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:27.651328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:27.652794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:19:27.664199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:27.665425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:27.666646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 617/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:19:27.671367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:27.775755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02221e-d0d6-4031-ac5a-1c8cbe808a59 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:27.809904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:27.810499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:27.810777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:27.811039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:27.811747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:27.812043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:27.813482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:27.830896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 623/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 00:19:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:27.837453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:27.937414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9494-fb1b-4c3d-8422-93452e4af93c None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:27.971054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:27.971659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:27.971947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:27.972197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:28.034583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:28.041335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:28.042829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:28.142674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 618/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:19:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:19:28.148804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:28.177331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:28.254992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452d56-4dcf-4ea5-94e6-c0d73ce4c13a None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:28.290644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:28.291214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:28.291494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:28.291766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:28.301798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:28.303235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:28.318864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:28.330466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:28.331683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:28.331683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:28.331683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:28.331683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 624/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:28.331956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:28.421612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:28.427291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc0b3-7fd9-4982-b2c0-1929b597c387 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:28.463321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:28.463989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:28.464232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:28.464530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:28.465172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:28.465478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:28.466849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:28.479029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 619/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:28.485290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:28.568515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:28.590816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ff3ed-f0f5-4866-b0ad-51cf1b3b7c31 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:28.625382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:28.626207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:28.626726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:28.627037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:28.634890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:28.636414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:19:28.647440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:28.648532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:28.654636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 625/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:19:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:28.655469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:28.755254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141b5ca-6e01-40c3-9b32-179e158f0c71 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:28.793269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:28.793864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:28.794217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:28.794481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:28.795189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:28.795487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:28.796940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a8e3-d6f6-4011-93cc-6cb8836a558e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:28.808791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 620/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:28.815381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:28.916967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf27ed-44e9-48b0-9ff8-f19f957030a2 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:28.951216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:28.951814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:28.952086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:28.952379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:28.953181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] RBAC: Adding query filter params (role.id=deb6dc5f801e411882108de755caff96, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:28.953500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:28.954888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94927055-5094-40a9-806e-95c23a9b4e4e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:28.961488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 626/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:19:28 2022] GET /identity/v3/role_assignments?user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=deb6dc5f801e411882108de755caff96 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:31.047175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c830-ec9b-420d-aa3f-a7cd2c7a6081 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:31.047738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c830-ec9b-420d-aa3f-a7cd2c7a6081 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:31.047738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c830-ec9b-420d-aa3f-a7cd2c7a6081 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:31.049109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 621/1247] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:31.055901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:31.056477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:31.056617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:31.063203 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:31.085002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:31.087153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:31.173460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 627/1248] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:31.449528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c830-ec9b-420d-aa3f-a7cd2c7a6081 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:31.468163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:31.555827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c830-ec9b-420d-aa3f-a7cd2c7a6081 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:31.595892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:31.596580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:31.596857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:31.597122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:31.607619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:19:31.612577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:31.613770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:31.614977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 622/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:31.620594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:31.722612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3843a-48e7-4f30-8926-90b9cd28cd7c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:31.752434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:31.758178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:31.758791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:31.759067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:31.759334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:31.759993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:31.760294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:31.761765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:31.770101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 628/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:31.776170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:31.853502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:31.876306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639f52d-4a52-4e1f-a23c-38093f46ddd5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:31.909592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:31.910211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:31.910484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:31.910752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:31.921567 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:31.923013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:31.938283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:31.939396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:31.940286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:31.940286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:31.940286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:31.940286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 623/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:31.945640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:32.047956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b97c0-a5f1-41fc-8484-5241f9abb588 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:32.082363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:32.083023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:32.083229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:32.083507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:32.084143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:32.084475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:32.085972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:32.098008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 629/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:32.104430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:32.204661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf87db-bc1a-4e0b-be5a-08ae9ac1e8b5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:32.231712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:32.239386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:32.240015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:32.240294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:32.240608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] PATH_INFO: `/v3/users/placement` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:32.249343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:32.250802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:19:32.261970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:32.263655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:32.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 624/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:32.270632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:32.328273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:32.375552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc743013-e7bb-4231-ac3c-15c3ff93ea32 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:32.410487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:32.411085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:32.411361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:32.411627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:32.412849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:32.413737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:32.415183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:32.432697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 630/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 00:19:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:32.438863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:32.540490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426c0db-bbe6-48fc-a30d-e09908cc4abf None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:32.574139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:32.574708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:32.574987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:32.575256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:32.629911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:32.643222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:32.644748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:32.755661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 625/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:19:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:19:32.762070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:32.865352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734bf18-44aa-40d0-a520-b56d6829724c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:32.879104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:32.901200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:32.901755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:32.902032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:32.902306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:32.912043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:32.913685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:32.928723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:32.930000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:32.931123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:32.931123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:32.931123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:32.931123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 631/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:32.935890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:33.027454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:33.039223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a841b-0394-4a5c-9892-8021e7516776 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:33.073909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:33.074519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:33.074791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:33.075059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:33.075686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:33.075983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:33.077456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:33.088550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 626/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:33.094378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:33.199180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b13c-dfee-4f91-85fd-d51a4a8de01c None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:33.239555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:33.240197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:33.240572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:33.240848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:33.249013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:33.250452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:19:33.262782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:33.268100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:33.269269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 632/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:33.270300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:33.376147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9291-2198-4c0b-aff2-d51e613bd4f5 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:33.410604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:33.411219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:33.411496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:33.411762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:33.412488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:33.412792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:33.415192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6ae8-ad0a-4398-b1f2-067999a4af87 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:33.427951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 627/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:33.434580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:33.462522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:33.542892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9ed1-c911-4b3e-b258-09d09f35ff2d None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:33.577376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:33.577962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:33.578237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:33.578502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:33.579219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:33.579548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:33.580991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247432e-abc3-4924-8f37-4b77c5e8aa74 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:33.587204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 633/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:19:33 2022] GET /identity/v3/role_assignments?user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=ac2134ae61074ce6b751deadce042c42 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:35.507417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bda3f-b76a-4c2f-8e0e-32ba2f6408fc None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:35.507965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bda3f-b76a-4c2f-8e0e-32ba2f6408fc None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:35.508057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bda3f-b76a-4c2f-8e0e-32ba2f6408fc None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:35.509399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 628/1261] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:35.515753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:35.522776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:35.522776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:35.536020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:35.537269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:35.607361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 634/1262] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:35.882640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bda3f-b76a-4c2f-8e0e-32ba2f6408fc None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:35.922451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:35.995414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bda3f-b76a-4c2f-8e0e-32ba2f6408fc None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:36.033254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:36.033884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:36.034164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:36.034436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:36.047198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:19:36.052260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:36.053520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:36.055046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 629/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:19:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:36.060288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:36.079101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:36.171058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e519b-67ce-4944-9459-9097b8be336a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:36.214213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:36.214828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:36.215099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:36.215432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:36.216024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:36.216342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:36.217707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:36.228102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 635/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:19:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:19:36.233812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:36.308259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:36.344049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eceb5-8475-4694-b7cd-9c212de9b3f9 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:36.379076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:36.379844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:36.380122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:36.380411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:36.391382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:36.392874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:36.408298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:36.409475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:36.410515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:36.410515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:36.410515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:36.410515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 630/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:36.416274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:36.508700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:36.528801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5a804-863a-41e0-8dc9-cfd776d5bb2e None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:36.566285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:36.566886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:36.567159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:36.567424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:36.568177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:36.568386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:36.570310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:36.582491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 636/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:36.591952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:36.703460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5449d2-cc21-4c25-b686-ecb54638438e None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:36.724482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:36.739237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:36.739785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:36.740071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:36.740408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:36.753107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:36.755257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:19:36.776461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:36.778658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:36.779863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 631/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:36.783853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:36.891340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c786556-ec16-4ffe-984f-d7fd70b33237 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:36.927739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:36.928465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:36.929298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:36.929565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:36.930296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:36.930599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:36.932028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:36.943646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:36.953906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 637/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 00:19:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:36.961096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:37.067289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2704d9-2642-4ed5-b0ec-c7826b3ab839 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:37.104780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:37.105534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:37.105710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:37.106021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:37.117458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:37.119082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:37.133350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:37.134584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:37.135789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:37.135789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:37.135789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:37.136497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 632/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:37.140513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:37.259328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a1059-5000-44fa-a007-2eda7b7630cf None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:37.302964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:37.304153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:37.304489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:37.304756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:37.305420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:37.305724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:37.307120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:37.319152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 638/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:37.325685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:37.365783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:37.445881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88e6d4-051c-4eb5-8475-ae7119ef1854 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:37.487676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:37.488376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:37.488659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:37.488924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:37.496836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:37.498286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:19:37.510809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:37.512070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:37.513988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 633/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:37.518254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:37.592653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:37.641770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee28228-b489-40f9-8268-4a84d5cb7cfa None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:37.677115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:37.677730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:37.678013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:37.678337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:37.679171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:37.679612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:37.681147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b144fe-cd8a-41fd-a1c4-12834be335bb None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:37.697002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 639/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:37.705671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:37.819648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:37.828432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b86c00-8115-4836-913c-9259c90afc77 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:37.864989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2d74c-c85d-4d8d-8abc-3c419003f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:37.865543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2d74c-c85d-4d8d-8abc-3c419003f7a6 None admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:37.865818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2d74c-c85d-4d8d-8abc-3c419003f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:37.866085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2d74c-c85d-4d8d-8abc-3c419003f7a6 None admin] PATH_INFO: `/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/42f4dfa35f464a3f9cdbbc1e1ee4a980/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:37.868446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2d74c-c85d-4d8d-8abc-3c419003f7a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=84d7e2df97aa4867a2b084bd767c48f5, user_id=42f4dfa35f464a3f9cdbbc1e1ee4a980, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:37.869895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2d74c-c85d-4d8d-8abc-3c419003f7a6 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:37.965020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 634/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 00:19:37 2022] PUT /identity/v3/projects/84d7e2df97aa4867a2b084bd767c48f5/users/42f4dfa35f464a3f9cdbbc1e1ee4a980/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:19:39.933035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931353-3ad0-49d2-bb3e-465dc7b6f14c None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:39.933563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931353-3ad0-49d2-bb3e-465dc7b6f14c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:39.933686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931353-3ad0-49d2-bb3e-465dc7b6f14c None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:39.935387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 640/1274] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:39.940794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:39.941476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:39.941695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:39.960927 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:39.962157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:40.028757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 635/1275] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:40.297435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931353-3ad0-49d2-bb3e-465dc7b6f14c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:40.400348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931353-3ad0-49d2-bb3e-465dc7b6f14c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:40.418020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:40.435650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:40.436268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:40.436568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:40.436833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:40.446994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:19:40.451656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:40.452799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:40.453826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 641/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 00:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:40.459007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:40.541145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:40.559840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e9232-779f-4c18-8d7c-02ce9de40301 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:40.593835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:40.594435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:40.594710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:40.594975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:40.595660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:40.595920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:40.597313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:40.605325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 636/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 00:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:40.610972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:40.711062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346e19c-1bc2-4569-a1d9-584840a31ce6 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:40.744444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:40.745084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:40.745354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:40.745617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:40.755554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:40.757829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:40.771904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:40.778217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:40.779286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:40.779286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:40.779286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:40.779286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 642/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:40.779732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:40.881862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1c10e-a3b2-4313-a760-f695feb65f35 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:40.915847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:40.916472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:40.916752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:40.917024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:40.917661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:40.917963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:40.919333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:40.930977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 637/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:40.937182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:40.968935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:41.037656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38451c-8f52-4776-9b3b-868e8d15a9ed None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:41.072921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:41.073544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:41.073825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:41.074095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:41.082431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:41.083864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:19:41.095649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:41.100681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:41.102048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 643/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:41.102838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:41.106631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:41.204314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde48c45-b466-49c6-bf89-594cea445c79 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:41.238754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:41.239349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:41.239622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:41.239889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:41.240633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:41.240935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:41.242650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:41.261161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 638/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 00:19:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:41.267177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:41.358333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:41.368200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45d1d-bc28-48d8-b0fd-dc50120c2757 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:41.401722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:41.402319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:41.402592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:41.402857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:41.473060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:41.474455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:41.565005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:41.576104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 644/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 00:19:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:19:41.582131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:41.645407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:41.688802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1da946-11fa-4b70-81c6-a5f17ad9abfb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:41.728254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:41.728831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:41.729107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:41.729377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:41.739478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:41.741096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:19:41.756739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:41.757932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:19:41.758845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:19:41.758845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:19:41.758845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:41.758845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 639/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:41.763921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:41.864992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f568990-efa7-4471-9e97-53c4349c44e2 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:41.901015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:41.901719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:41.902042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:41.902359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:41.903101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:41.903464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:41.904981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:41.917310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 645/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 00:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:41.923188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:42.031841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f26d3-4ee7-40f0-8f98-c085de0f7846 None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:42.047197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:42.073032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:42.073964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:42.074237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:42.074533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:42.081578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:42.083022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:19:42.094570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:42.099829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: ERROR keystone.server.flask.application Aug 31 00:19:42.101117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 640/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 00:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:42.102456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:42.154398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:42.206106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332338a7-d011-4840-afa0-bc147303ce27 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:42.239149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:42.239747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:42.240020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:42.240284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:42.241007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:42.241888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:42.243363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af61a4e-a86b-40e1-a4e1-f74c85539b4a None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:42.256025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 646/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 00:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:42.262793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:42.365745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed72d2-b2d2-4810-959f-5c4ea4ebe96b None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:42.399197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:42.399793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:42.400065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:42.400357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:42.401067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] RBAC: Adding query filter params (role.id=ac2134ae61074ce6b751deadce042c42, scope.project.id=84d7e2df97aa4867a2b084bd767c48f5, user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:42.401370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:42.402834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac7a-0892-4c1e-b46b-9403a70d8bbb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:42.408658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 641/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 00:19:42 2022] GET /identity/v3/role_assignments?user.id=42f4dfa35f464a3f9cdbbc1e1ee4a980&scope.project.id=84d7e2df97aa4867a2b084bd767c48f5&role.id=ac2134ae61074ce6b751deadce042c42 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:44.353284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a731735-5f82-4a1d-bdf5-e9ef84cd749a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:44.353820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a731735-5f82-4a1d-bdf5-e9ef84cd749a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:44.353820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a731735-5f82-4a1d-bdf5-e9ef84cd749a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:44.355510 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 647/1288] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:44.360703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:44.361197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:44.361328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:44.379845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:44.381065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:44.412705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:44.453293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 642/1289] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:19:44.724300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a731735-5f82-4a1d-bdf5-e9ef84cd749a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:44.829802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a731735-5f82-4a1d-bdf5-e9ef84cd749a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:44.841559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:44.864525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:44.865139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:44.865411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:44.865674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:44.866856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:44.868191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:44.877540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:44.878816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:44.879907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 648/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:19:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:19:44.885387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:44.989670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f63367-f20c-489d-8f21-200f9c41597a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:45.026388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:45.026999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:45.027280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:45.027553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:45.028184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] RBAC: Adding query filter params (name=placement) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:45.028517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:45.029970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13314910-99f7-49c8-a924-1e8444b2073c None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:45.034787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 643/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 00:19:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:45.041111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:45.145380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48411c34-4946-4fb6-9805-91c7e85b16e9 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:45.179832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:45.180239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:45.180548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:45.180822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] PATH_INFO: `/v3/services` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:45.181452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] RBAC: Adding query filter params (type=placement) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:45.181754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:45.183185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abb05c-799a-46a7-93b5-b1578ac6f15c None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:45.188255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 649/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 00:19:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:47.122291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4589db-3956-43b3-816d-262f20335fe9 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:47.122858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4589db-3956-43b3-816d-262f20335fe9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:47.122858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4589db-3956-43b3-816d-262f20335fe9 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:47.124175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 644/1293] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:47.129413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab673-f86e-4c43-8cec-fc6a9fda4bf7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:47.129843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab673-f86e-4c43-8cec-fc6a9fda4bf7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:47.129991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab673-f86e-4c43-8cec-fc6a9fda4bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:47.149572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-374ab673-f86e-4c43-8cec-fc6a9fda4bf7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:47.150760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-374ab673-f86e-4c43-8cec-fc6a9fda4bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:47.179760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:47.217744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 650/1294] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:47.489878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4589db-3956-43b3-816d-262f20335fe9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:47.525054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:47.595721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4589db-3956-43b3-816d-262f20335fe9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:47.630693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d51f9-6666-44be-9b10-c2f62a417b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:47.631291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d51f9-6666-44be-9b10-c2f62a417b74 None admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:47.632250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d51f9-6666-44be-9b10-c2f62a417b74 None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:47.632571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d51f9-6666-44be-9b10-c2f62a417b74 None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:47.633233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d51f9-6666-44be-9b10-c2f62a417b74 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:47.634618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d51f9-6666-44be-9b10-c2f62a417b74 None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:47.706524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 645/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 00:19:47 2022] POST /identity/v3/services => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:19:49.619874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6c7c7-288a-4687-819d-c358bd86a6bc None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:49.620412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6c7c7-288a-4687-819d-c358bd86a6bc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:49.620412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6c7c7-288a-4687-819d-c358bd86a6bc None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:49.621781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 651/1296] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:49.626851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:49.627245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:49.627401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:49.646916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:49.648192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:49.722763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 646/1297] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:49.992528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6c7c7-288a-4687-819d-c358bd86a6bc None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:50.095866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6c7c7-288a-4687-819d-c358bd86a6bc None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:50.129422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:50.130048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:50.130326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:50.130595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] PATH_INFO: `/v3/services/placement` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:50.131168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:50.133264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:50.142094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:50.143292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:50.144143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 652/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:19:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:50.149961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:50.168211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:50.255205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4187377-5d58-4444-964f-ab05aa1cf056 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:50.292747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:50.293366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:50.293646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:50.293918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:50.294546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] RBAC: Adding query filter params (name=placement) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:50.294845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:50.296225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:50.301710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 647/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 00:19:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:50.308440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:50.345647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:50.416519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b557a49-9410-4764-8b1f-616f6fc3b1db None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:50.452468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:50.453073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:50.453350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:50.453619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:50.454307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] RBAC: Adding query filter params (interface=public, service_id=31ec1f5232b347cdbec6e1e600a1c089, region_id=RegionOne) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:50.454611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:50.455975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b21db-ad2b-4a0e-9b6b-c8f020fd3749 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:50.462102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 653/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 00:19:50 2022] GET /identity/v3/endpoints?service_id=31ec1f5232b347cdbec6e1e600a1c089&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:50.467119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:50.557783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:50.576427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb230419-f2c0-4225-ad78-8c3a80f01ffb None admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:50.611877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7ef3d-6f7d-45dd-a153-e86e4256c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:50.612510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7ef3d-6f7d-45dd-a153-e86e4256c4fc None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:50.612794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7ef3d-6f7d-45dd-a153-e86e4256c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:50.613066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7ef3d-6f7d-45dd-a153-e86e4256c4fc None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:50.613668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7ef3d-6f7d-45dd-a153-e86e4256c4fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:50.615039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7ef3d-6f7d-45dd-a153-e86e4256c4fc None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:50.622662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 648/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 00:19:50 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:19:52.531621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3737c11-2627-4c7e-9b83-c14f8b8fb724 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:52.532287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3737c11-2627-4c7e-9b83-c14f8b8fb724 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:52.532287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3737c11-2627-4c7e-9b83-c14f8b8fb724 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:52.533574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 654/1302] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:19:52.539355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:52.539745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:52.539907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:52.559070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:19:52.560256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:19:52.627448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 649/1303] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:19:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:19:52.901854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3737c11-2627-4c7e-9b83-c14f8b8fb724 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:53.001686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:53.010711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3737c11-2627-4c7e-9b83-c14f8b8fb724 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:53.046496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:53.047121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:53.047401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:53.047672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:53.048248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:53.049680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:19:53.059426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:19:53.060648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application with Lock( Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: ERROR keystone.server.flask.application Aug 31 00:19:53.061516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 655/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 00:19:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:19:53.066793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:53.143761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:53.166326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d527139-cf13-4b57-9cbb-685e1a743142 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:53.201717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:53.202321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:53.202602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:53.202875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] PATH_INFO: `/v3/services` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:53.203496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] RBAC: Adding query filter params (name=placement) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:19:53.203797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:53.205191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef107d3-24cf-4eb4-9fd7-f39fa8d7dadf None admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:53.210470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 650/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 00:19:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:19:53.216790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:19:53.322672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffd00a-ca35-41e2-9bbb-82a67336db47 None admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:19:53.356775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6d097-aeed-4162-bf1b-123edd35902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:19:53.357376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d097-aeed-4162-bf1b-123edd35902d None admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:19:53.357656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d097-aeed-4162-bf1b-123edd35902d None admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:19:53.357956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6d097-aeed-4162-bf1b-123edd35902d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:19:53.358633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6d097-aeed-4162-bf1b-123edd35902d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:19:53.359990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6d097-aeed-4162-bf1b-123edd35902d None admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:19:53.427934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 656/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:19:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:03.207262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=30 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:03.423011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:20.983455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc7d8e-12de-4772-a8d7-af5f5d22bd75 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:20.984039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc7d8e-12de-4772-a8d7-af5f5d22bd75 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:20.984039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc7d8e-12de-4772-a8d7-af5f5d22bd75 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:20.985625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 651/1307] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:20.990876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa494e5b-5b50-41a6-b5f2-57e4bbf17388 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:20.991401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa494e5b-5b50-41a6-b5f2-57e4bbf17388 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:20.991491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa494e5b-5b50-41a6-b5f2-57e4bbf17388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:21.107388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fa494e5b-5b50-41a6-b5f2-57e4bbf17388 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:21.108877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fa494e5b-5b50-41a6-b5f2-57e4bbf17388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:21.272914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 657/1308] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:26:21.589904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a3f27-94d0-4b40-9962-7a760c3bb69a None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:21.590282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a3f27-94d0-4b40-9962-7a760c3bb69a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:21.590449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a3f27-94d0-4b40-9962-7a760c3bb69a None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:21.592220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 652/1309] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 31 00:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:21.600069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148be0f-7d7e-4b25-999a-0eae46f07d24 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:21.600377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148be0f-7d7e-4b25-999a-0eae46f07d24 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:21.600665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148be0f-7d7e-4b25-999a-0eae46f07d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:21.712158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9148be0f-7d7e-4b25-999a-0eae46f07d24 None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:21.714393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9148be0f-7d7e-4b25-999a-0eae46f07d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:21.739032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:21.831812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 658/1310] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 31 00:26:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:21.832529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a3f27-94d0-4b40-9962-7a760c3bb69a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:21.834475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fe3a3f27-94d0-4b40-9962-7a760c3bb69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:21.971588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a3f27-94d0-4b40-9962-7a760c3bb69a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:22.097755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:22.098476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:22.098769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:22.099313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:22.101145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:22.176057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:22.286092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:22.287777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50e077-5d98-47a2-a7aa-287a44fee5b4 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:22.316936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 653/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:24.886346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f7dc-ebc1-4618-9f2b-10131afebfa0 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:24.887187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f7dc-ebc1-4618-9f2b-10131afebfa0 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:24.887187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f7dc-ebc1-4618-9f2b-10131afebfa0 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:24.888596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 659/1312] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:24.893832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d2ba2-9189-42a4-810a-e89148cd26ea None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:24.894493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d2ba2-9189-42a4-810a-e89148cd26ea None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:24.894493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d2ba2-9189-42a4-810a-e89148cd26ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:24.989050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c89d2ba2-9189-42a4-810a-e89148cd26ea None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:24.990284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c89d2ba2-9189-42a4-810a-e89148cd26ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:25.103223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 654/1313] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:25.393968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f7dc-ebc1-4618-9f2b-10131afebfa0 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:25.395627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b920f7dc-ebc1-4618-9f2b-10131afebfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:25.524481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f7dc-ebc1-4618-9f2b-10131afebfa0 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:25.612500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:25.617373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:25.618896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:25.619190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:25.619468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:25.622209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:25.755957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:25.757447 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2d0f7-40b7-4687-be37-b286f2c157e9 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:25.777853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 660/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:28.337045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512554af-378f-4dd2-b6d4-5c77363ae5cd None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:28.337631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512554af-378f-4dd2-b6d4-5c77363ae5cd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:28.337631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512554af-378f-4dd2-b6d4-5c77363ae5cd None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:28.339512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 655/1315] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:28.347847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bdf32-2507-4098-949d-223cf8887e00 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:28.348382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bdf32-2507-4098-949d-223cf8887e00 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:28.348832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bdf32-2507-4098-949d-223cf8887e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:28.431784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e13bdf32-2507-4098-949d-223cf8887e00 None None] MFA Rules not processed for user `4e79330e00e34ea08d7152d870440c86`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:28.433812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e13bdf32-2507-4098-949d-223cf8887e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:28.495664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:28.542952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 661/1316] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:28.843925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512554af-378f-4dd2-b6d4-5c77363ae5cd None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:28.845526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-512554af-378f-4dd2-b6d4-5c77363ae5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:28.856823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:28.954941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512554af-378f-4dd2-b6d4-5c77363ae5cd None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:29.032129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:29.032822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:29.033107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:29.033374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:29.035019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:29.101199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:29.166070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:29.167701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d62e3b-be7a-4fbe-a059-7846abbb329a service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:29.185958 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 656/1317] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:32.188355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc0a8-95b3-4687-b6e3-5aa0fc79f485 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:32.188898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc0a8-95b3-4687-b6e3-5aa0fc79f485 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:32.188898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bc0a8-95b3-4687-b6e3-5aa0fc79f485 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:32.190221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 662/1318] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:32.195624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df1fc3-9fbb-479e-bc1f-6847a63d31ef None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:32.196017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df1fc3-9fbb-479e-bc1f-6847a63d31ef None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:32.196167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df1fc3-9fbb-479e-bc1f-6847a63d31ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:32.264575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a4df1fc3-9fbb-479e-bc1f-6847a63d31ef None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:32.265775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a4df1fc3-9fbb-479e-bc1f-6847a63d31ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:32.362079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 657/1319] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:32.675434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964c2b8-d511-452a-b0f6-d72f2d3f1f49 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:32.677616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964c2b8-d511-452a-b0f6-d72f2d3f1f49 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:32.677616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964c2b8-d511-452a-b0f6-d72f2d3f1f49 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:32.677616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 663/1320] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 31 00:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:32.683556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c12b60-e082-46c6-89c8-99ad1a85153b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:32.683779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c12b60-e082-46c6-89c8-99ad1a85153b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:32.684049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c12b60-e082-46c6-89c8-99ad1a85153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:32.706146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:32.765038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-77c12b60-e082-46c6-89c8-99ad1a85153b None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:32.766263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-77c12b60-e082-46c6-89c8-99ad1a85153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:32.854341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 658/1321] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 31 00:26:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:32.863690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964c2b8-d511-452a-b0f6-d72f2d3f1f49 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:32.865503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9964c2b8-d511-452a-b0f6-d72f2d3f1f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:32.974122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964c2b8-d511-452a-b0f6-d72f2d3f1f49 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:33.049189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:33.049738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:33.050018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:33.050278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:33.051891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:33.059757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:33.167062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:33.168627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479c947-dbe1-45a7-b535-59ed608deaa8 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:33.188018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 664/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:36.845091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254eae1-0baf-4f6d-a756-1bb077e99b98 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:36.845680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254eae1-0baf-4f6d-a756-1bb077e99b98 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:36.845680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254eae1-0baf-4f6d-a756-1bb077e99b98 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:36.847018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 659/1323] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:36.852770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdde5c-ffec-4b52-8860-113368185fd7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:36.853225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdde5c-ffec-4b52-8860-113368185fd7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:36.853342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdde5c-ffec-4b52-8860-113368185fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:36.937341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-4afdde5c-ffec-4b52-8860-113368185fd7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:36.939247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4afdde5c-ffec-4b52-8860-113368185fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:37.007698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:37.054523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 665/1324] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:37.378226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254eae1-0baf-4f6d-a756-1bb077e99b98 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:37.379795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9254eae1-0baf-4f6d-a756-1bb077e99b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:37.500762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254eae1-0baf-4f6d-a756-1bb077e99b98 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:37.514471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:37.580793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:37.581402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:37.581679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:37.581973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:37.583568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:37.702905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:37.704249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbda36b-dfc4-4e57-8678-79925b3f12cc service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:37.725143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 660/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:39.752686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f94b-0952-4e2f-8a04-1715d4f72652 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:39.753210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f94b-0952-4e2f-8a04-1715d4f72652 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:39.753328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f94b-0952-4e2f-8a04-1715d4f72652 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:39.754719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 666/1326] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:39.760281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e33a4-5e26-42f4-8cea-8572408d2a5a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:39.760766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e33a4-5e26-42f4-8cea-8572408d2a5a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:39.760960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e33a4-5e26-42f4-8cea-8572408d2a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:39.818021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:39.841009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6b2e33a4-5e26-42f4-8cea-8572408d2a5a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:39.842240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6b2e33a4-5e26-42f4-8cea-8572408d2a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:39.962033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 661/1327] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:40.248586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f94b-0952-4e2f-8a04-1715d4f72652 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:40.251002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a187f94b-0952-4e2f-8a04-1715d4f72652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:40.361794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f94b-0952-4e2f-8a04-1715d4f72652 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:40.446950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:40.447915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:40.448191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:40.448502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:40.450133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:40.468910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:40.572645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:40.574104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7d3ca-4c45-4fb6-bc91-60bcd106ba9a service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:40.594398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 667/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:42.677850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932e3f-f0b7-4e0f-881c-e473107c625e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:42.678539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932e3f-f0b7-4e0f-881c-e473107c625e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:42.678539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932e3f-f0b7-4e0f-881c-e473107c625e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:42.679759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 662/1329] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:42.685783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fc34d-ac7a-41ec-afe7-260175d49ca6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:42.686273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fc34d-ac7a-41ec-afe7-260175d49ca6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:42.686354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fc34d-ac7a-41ec-afe7-260175d49ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:42.760623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ab3fc34d-ac7a-41ec-afe7-260175d49ca6 None None] MFA Rules not processed for user `4e79330e00e34ea08d7152d870440c86`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:42.761830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ab3fc34d-ac7a-41ec-afe7-260175d49ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:42.828790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:42.862846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 668/1330] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:43.153226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932e3f-f0b7-4e0f-881c-e473107c625e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:43.154793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-51932e3f-f0b7-4e0f-881c-e473107c625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:43.271305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932e3f-f0b7-4e0f-881c-e473107c625e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:43.292446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:43.348991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:43.349597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:43.349874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:43.350144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:43.351744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:43.470679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:43.472982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0c643-4c65-489a-8f08-083ba008eaa6 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:43.496264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 663/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:45.944074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a30280-7bc3-42f6-b4c3-666e43dd9af5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:45.944670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a30280-7bc3-42f6-b4c3-666e43dd9af5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:45.944769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a30280-7bc3-42f6-b4c3-666e43dd9af5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:45.946137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 669/1332] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:45.952612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9afe2-2cc5-4863-aead-db1ffd234595 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:45.953096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9afe2-2cc5-4863-aead-db1ffd234595 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:45.953193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9afe2-2cc5-4863-aead-db1ffd234595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:46.023687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:46.027739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-29e9afe2-2cc5-4863-aead-db1ffd234595 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:46.029073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-29e9afe2-2cc5-4863-aead-db1ffd234595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:46.130866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 664/1333] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:46.412314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a30280-7bc3-42f6-b4c3-666e43dd9af5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:46.413919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c9a30280-7bc3-42f6-b4c3-666e43dd9af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:46.522741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a30280-7bc3-42f6-b4c3-666e43dd9af5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:46.597570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:46.598174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:46.598439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:46.598720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:46.600351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:46.627934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:46.714460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:46.715894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1fc3c-0c63-4d7a-b972-667ffa60ed1f service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:46.734578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 670/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:48.729926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf7ba9-a61b-4a55-87ca-0fb745dd70eb None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:48.730520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf7ba9-a61b-4a55-87ca-0fb745dd70eb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:48.730520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf7ba9-a61b-4a55-87ca-0fb745dd70eb None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:48.731791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 665/1335] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:48.737182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2683a-70b7-4443-84cc-fdb2ebe52746 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:48.737583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2683a-70b7-4443-84cc-fdb2ebe52746 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:48.737775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2683a-70b7-4443-84cc-fdb2ebe52746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:48.805404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ded2683a-70b7-4443-84cc-fdb2ebe52746 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:48.807497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ded2683a-70b7-4443-84cc-fdb2ebe52746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:48.883950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:48.907988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 671/1336] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:49.196144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf7ba9-a61b-4a55-87ca-0fb745dd70eb None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:49.197802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-28bf7ba9-a61b-4a55-87ca-0fb745dd70eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:49.304824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf7ba9-a61b-4a55-87ca-0fb745dd70eb None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:49.333055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:49.380230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:49.380860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:49.381128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:49.381398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:49.383010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:49.492089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:49.493631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b3800-0282-4c57-8016-5b8cf3f722e0 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:49.512454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 666/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:52.131892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206747f-800d-4af2-8c29-e1b4e312951a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:52.132525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206747f-800d-4af2-8c29-e1b4e312951a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:52.132525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206747f-800d-4af2-8c29-e1b4e312951a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:52.133846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 672/1338] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:52.139231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb1cae-4766-474d-9dbd-daecff71dade None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:52.139699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb1cae-4766-474d-9dbd-daecff71dade None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:52.139798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb1cae-4766-474d-9dbd-daecff71dade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:52.208693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-42bb1cae-4766-474d-9dbd-daecff71dade None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:52.209950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-42bb1cae-4766-474d-9dbd-daecff71dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:52.218678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:52.312142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 667/1339] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:52.604367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206747f-800d-4af2-8c29-e1b4e312951a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:52.605938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0206747f-800d-4af2-8c29-e1b4e312951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:52.734715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206747f-800d-4af2-8c29-e1b4e312951a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:52.821039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:52.822341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:52.822341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:52.822341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:52.823724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:52.868924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:52.952499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:52.953933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463509b-b3bb-4cee-802b-6b667df716c0 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:52.973987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 673/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:57.372401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fe5c7-01ba-49ad-a67b-4c1c5f3d1cca None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:57.372930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fe5c7-01ba-49ad-a67b-4c1c5f3d1cca None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:57.372930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fe5c7-01ba-49ad-a67b-4c1c5f3d1cca None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:57.374271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 668/1341] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:26:57.379887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfc395-e22f-4327-8543-58e19e791c15 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:57.380372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfc395-e22f-4327-8543-58e19e791c15 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:57.380480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfc395-e22f-4327-8543-58e19e791c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:57.453174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a1cfc395-e22f-4327-8543-58e19e791c15 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:57.454406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a1cfc395-e22f-4327-8543-58e19e791c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:57.547993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:57.560122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 674/1342] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:57.849440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fe5c7-01ba-49ad-a67b-4c1c5f3d1cca None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:57.851051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3c1fe5c7-01ba-49ad-a67b-4c1c5f3d1cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:57.959814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fe5c7-01ba-49ad-a67b-4c1c5f3d1cca None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:57.999966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:58.037385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:58.038002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:58.038301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:58.038569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:58.041579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:58.161519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:58.162894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc20bd2-95d2-432a-a8cd-fe67475f4fc3 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:58.182484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 669/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:01.274819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e7759-ff16-4b2f-82b5-f4263b06ed09 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:01.275561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e7759-ff16-4b2f-82b5-f4263b06ed09 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:01.275796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e7759-ff16-4b2f-82b5-f4263b06ed09 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:01.277223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 675/1344] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:01.283034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e557f-be80-4c57-9339-7464d39b9e66 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:01.283493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e557f-be80-4c57-9339-7464d39b9e66 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:01.283752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e557f-be80-4c57-9339-7464d39b9e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:01.369429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ad4e557f-be80-4c57-9339-7464d39b9e66 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:01.370667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ad4e557f-be80-4c57-9339-7464d39b9e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:01.391687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:01.493194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 670/1345] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:01.785641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e7759-ff16-4b2f-82b5-f4263b06ed09 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:01.787182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4f8e7759-ff16-4b2f-82b5-f4263b06ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:01.908920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e7759-ff16-4b2f-82b5-f4263b06ed09 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:01.999025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:01.999652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:01.999930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:02.000201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:02.002304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:02.053735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:02.127106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:02.128994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39713e58-040d-442e-a983-6d6464d9c5e5 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:02.151642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 676/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:06.963648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2217-a9b6-469a-a536-b881c257168b None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:06.964205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2217-a9b6-469a-a536-b881c257168b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:06.964205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2217-a9b6-469a-a536-b881c257168b None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:06.965900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 671/1347] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:06.971752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0642e6d-6887-4389-b97b-5770e0ff53a8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:06.972203 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0642e6d-6887-4389-b97b-5770e0ff53a8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:06.972289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0642e6d-6887-4389-b97b-5770e0ff53a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:07.076605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c0642e6d-6887-4389-b97b-5770e0ff53a8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:07.078397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c0642e6d-6887-4389-b97b-5770e0ff53a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:07.217604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:07.221449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 677/1348] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:07.517473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2217-a9b6-469a-a536-b881c257168b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:07.519146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-931a2217-a9b6-469a-a536-b881c257168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:07.655351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2217-a9b6-469a-a536-b881c257168b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:07.726644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:07.755690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:07.756296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:07.756606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:07.756869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:07.758569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:07.897715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:07.899438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957a473-bf6b-44a6-a427-e179a7d7a1ee service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:07.925947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 672/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:13.357313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a53e8f-8885-4849-8648-434e352b7e40 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:13.357891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a53e8f-8885-4849-8648-434e352b7e40 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:13.358472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a53e8f-8885-4849-8648-434e352b7e40 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:13.361503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 678/1350] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:27:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:13.370401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ac9c-6a6a-41e0-af80-142d664ae1d7 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:13.371245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ac9c-6a6a-41e0-af80-142d664ae1d7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:13.371691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610ac9c-6a6a-41e0-af80-142d664ae1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:13.522580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3610ac9c-6a6a-41e0-af80-142d664ae1d7 None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:13.524192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3610ac9c-6a6a-41e0-af80-142d664ae1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:13.546870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:13.696731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 673/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:19.178115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf031eae-3f0f-404d-bcba-e7fe59b9508c None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:19.178973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf031eae-3f0f-404d-bcba-e7fe59b9508c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:19.178973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf031eae-3f0f-404d-bcba-e7fe59b9508c None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:19.181003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 679/1352] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:19.186371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232bec1-1b88-4cfd-80b1-94af10e0a718 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:19.187505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232bec1-1b88-4cfd-80b1-94af10e0a718 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:19.188034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232bec1-1b88-4cfd-80b1-94af10e0a718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:19.333849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4232bec1-1b88-4cfd-80b1-94af10e0a718 None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:19.337047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4232bec1-1b88-4cfd-80b1-94af10e0a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:19.455671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:19.505361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 674/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:21.658175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d70889-a968-40ce-b158-24ee0cb08065 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:21.658953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d70889-a968-40ce-b158-24ee0cb08065 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:21.658953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d70889-a968-40ce-b158-24ee0cb08065 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:21.661029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 680/1354] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:27:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:27:21.669809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39717473-afe9-434c-9e25-5e09e15600f0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:21.670570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39717473-afe9-434c-9e25-5e09e15600f0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:21.670725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39717473-afe9-434c-9e25-5e09e15600f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:21.783049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-39717473-afe9-434c-9e25-5e09e15600f0 None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:21.784308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-39717473-afe9-434c-9e25-5e09e15600f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:21.922667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 675/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:27:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:27:25.446107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49224d-c6d4-4e19-956b-9d67ff3692e7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:25.452827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49224d-c6d4-4e19-956b-9d67ff3692e7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:25.452827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49224d-c6d4-4e19-956b-9d67ff3692e7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:25.452827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 681/1356] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:25.455698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b89089-0925-4e69-bbd7-8c2f394a6eae None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:25.456518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b89089-0925-4e69-bbd7-8c2f394a6eae None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:25.456836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b89089-0925-4e69-bbd7-8c2f394a6eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:25.525326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:25.571231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-02b89089-0925-4e69-bbd7-8c2f394a6eae None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:25.573357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-02b89089-0925-4e69-bbd7-8c2f394a6eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:25.706297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 676/1357] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:27:26.058072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e08b5-bf18-4fbb-a596-72fdbcebbfda None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:26.058557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e08b5-bf18-4fbb-a596-72fdbcebbfda None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:26.059433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e08b5-bf18-4fbb-a596-72fdbcebbfda None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:26.063588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 682/1358] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 00:27:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:26.088389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc808f44-e105-4308-b72a-5e6d3cb57782 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:26.088389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc808f44-e105-4308-b72a-5e6d3cb57782 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:26.088389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc808f44-e105-4308-b72a-5e6d3cb57782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:26.259529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-cc808f44-e105-4308-b72a-5e6d3cb57782 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:26.261926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cc808f44-e105-4308-b72a-5e6d3cb57782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:26.356084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:26.485457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 677/1359] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 00:27:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:26.494541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e08b5-bf18-4fbb-a596-72fdbcebbfda None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:26.497599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b48e08b5-bf18-4fbb-a596-72fdbcebbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:26.746753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e08b5-bf18-4fbb-a596-72fdbcebbfda None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:26.852060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:26.923800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:26.925346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:26.925840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:26.926310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:26.929456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:27.085906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:27.088022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b177f-f0d0-4919-8791-6bfa0dd2c73a service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:27.112669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 683/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:29.883727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d628-7765-40b8-9869-93c418e470ed None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:29.884581 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d628-7765-40b8-9869-93c418e470ed None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:29.884581 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3d628-7765-40b8-9869-93c418e470ed None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:29.886339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 678/1361] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:27:29.891357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61780-99f0-43cc-93f2-87b3d8adbcc3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:29.892069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61780-99f0-43cc-93f2-87b3d8adbcc3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:29.892069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61780-99f0-43cc-93f2-87b3d8adbcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:29.978753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-05e61780-99f0-43cc-93f2-87b3d8adbcc3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:29.980002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-05e61780-99f0-43cc-93f2-87b3d8adbcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:30.022781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:30.136382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 684/1362] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:27:30.482913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3d628-7765-40b8-9869-93c418e470ed None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:30.484649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e2c3d628-7765-40b8-9869-93c418e470ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:30.739408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3d628-7765-40b8-9869-93c418e470ed None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:30.760687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:30.955152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:30.956178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:30.956932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:30.957326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:30.960112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:31.086361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:31.088738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054d77-6546-496f-a254-1d1391c4acd8 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:31.127166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 679/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:33.828057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45348ac3-48f5-47d0-af54-8f6acadcecf7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:33.828672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45348ac3-48f5-47d0-af54-8f6acadcecf7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:33.828757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45348ac3-48f5-47d0-af54-8f6acadcecf7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:33.831034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 685/1364] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:33.842252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2900-4ca3-42a9-aff0-6681d2d467c4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:33.843109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2900-4ca3-42a9-aff0-6681d2d467c4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:33.843511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2900-4ca3-42a9-aff0-6681d2d467c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:33.862106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:33.931217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-342c2900-4ca3-42a9-aff0-6681d2d467c4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:33.932778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-342c2900-4ca3-42a9-aff0-6681d2d467c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:34.061820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 680/1365] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:34.421788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef868a-8de3-4a13-8f77-4a3ee2dd24fb None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:34.422098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef868a-8de3-4a13-8f77-4a3ee2dd24fb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:34.422431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef868a-8de3-4a13-8f77-4a3ee2dd24fb None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:34.424093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 686/1366] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 00:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:27:34.431849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62511f-9cb1-428f-b020-fc870a7cd80f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:34.432832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62511f-9cb1-428f-b020-fc870a7cd80f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:34.434885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62511f-9cb1-428f-b020-fc870a7cd80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:34.606349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-bf62511f-9cb1-428f-b020-fc870a7cd80f None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:34.609764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bf62511f-9cb1-428f-b020-fc870a7cd80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:34.663903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:34.807594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 681/1367] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:34.812586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef868a-8de3-4a13-8f77-4a3ee2dd24fb None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:34.814508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-53ef868a-8de3-4a13-8f77-4a3ee2dd24fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:34.999577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef868a-8de3-4a13-8f77-4a3ee2dd24fb None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:35.027524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:35.169395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:35.170008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:35.170280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:35.170868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:35.172781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:35.311466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:35.313755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545fdbd-6c17-46e6-8f3e-328dc882a4f3 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:35.348702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 687/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:35.715483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4006c-1a6f-41ca-aff7-9a0145999b2f None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:35.716310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4006c-1a6f-41ca-aff7-9a0145999b2f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:35.716656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4006c-1a6f-41ca-aff7-9a0145999b2f None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:35.718843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 682/1369] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:35.726111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53bc9f-b5a1-4650-bf00-9ad7159e1dc8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:35.726862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53bc9f-b5a1-4650-bf00-9ad7159e1dc8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:35.727471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53bc9f-b5a1-4650-bf00-9ad7159e1dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:35.787787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:35.852576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2c53bc9f-b5a1-4650-bf00-9ad7159e1dc8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:35.854187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2c53bc9f-b5a1-4650-bf00-9ad7159e1dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:36.010210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 688/1370] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:36.666184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66890c-c9ce-402c-9b27-4d99d30eb074 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:36.666464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66890c-c9ce-402c-9b27-4d99d30eb074 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:36.666992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66890c-c9ce-402c-9b27-4d99d30eb074 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:36.668858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 683/1371] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 31 00:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:36.677373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2440f-78e2-45c5-b459-ef5f08978362 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:36.677935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2440f-78e2-45c5-b459-ef5f08978362 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:36.678461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2440f-78e2-45c5-b459-ef5f08978362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:36.789548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-86a2440f-78e2-45c5-b459-ef5f08978362 None None] MFA Rules not processed for user `a765c1f74578435d8f29fbf959e5f087`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:36.790781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-86a2440f-78e2-45c5-b459-ef5f08978362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:36.863950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:36.922173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 689/1372] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 31 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:36.923950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66890c-c9ce-402c-9b27-4d99d30eb074 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:36.926787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4f66890c-c9ce-402c-9b27-4d99d30eb074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:37.086732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66890c-c9ce-402c-9b27-4d99d30eb074 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:37.182912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:37.254631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:37.255264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:37.255545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:37.255800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:37.259382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:37.409270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:37.410820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049d351-2d03-415f-b83a-b9faf3fe8c13 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:37.433591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 684/1373] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:37.783650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdf07-b0a3-4dcc-a0ff-881c2628525d None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:37.784462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdf07-b0a3-4dcc-a0ff-881c2628525d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:37.784462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdf07-b0a3-4dcc-a0ff-881c2628525d None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:37.785597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 690/1374] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:37.792660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375170-f169-4634-9ed2-baafe4d13cf9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:37.793967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375170-f169-4634-9ed2-baafe4d13cf9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:37.794247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375170-f169-4634-9ed2-baafe4d13cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:37.891246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e4375170-f169-4634-9ed2-baafe4d13cf9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:37.892718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e4375170-f169-4634-9ed2-baafe4d13cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:37.920919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:38.015373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 685/1375] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:27:38.367625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdf07-b0a3-4dcc-a0ff-881c2628525d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:38.369556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a55bdf07-b0a3-4dcc-a0ff-881c2628525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:38.575540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdf07-b0a3-4dcc-a0ff-881c2628525d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:38.637417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:38.709263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:38.709942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:38.710175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:38.710792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:38.713200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:38.830337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:38.831972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba549df-3d6f-4acd-9f72-62a59db2f289 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:38.853208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 691/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:41.075869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff81f1-6ebe-43f1-9b95-2381995c56b4 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:41.076377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff81f1-6ebe-43f1-9b95-2381995c56b4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:41.076461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff81f1-6ebe-43f1-9b95-2381995c56b4 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:41.077810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 686/1377] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:41.083631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082733c-f20a-45af-b573-49e2dc9dc024 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:41.084035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082733c-f20a-45af-b573-49e2dc9dc024 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:41.085256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082733c-f20a-45af-b573-49e2dc9dc024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:41.173628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d082733c-f20a-45af-b573-49e2dc9dc024 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:41.175796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d082733c-f20a-45af-b573-49e2dc9dc024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:41.184706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:41.309775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 692/1378] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:41.641329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff81f1-6ebe-43f1-9b95-2381995c56b4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:41.644198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7cff81f1-6ebe-43f1-9b95-2381995c56b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:41.898637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff81f1-6ebe-43f1-9b95-2381995c56b4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:41.903559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:42.067616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:42.068439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:42.068920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:42.069933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:42.072031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:42.204106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:42.205689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae56b9f-606c-473d-8578-a0d7c030e4da service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:42.227981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 687/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:44.645334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d445dc-6544-4a5a-ba7a-a3c2e6f0c3c5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:44.645907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d445dc-6544-4a5a-ba7a-a3c2e6f0c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:44.645907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d445dc-6544-4a5a-ba7a-a3c2e6f0c3c5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:44.648065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 693/1380] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:44.653161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501342d1-f635-437c-a42d-aa2de43b24bd None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:44.653626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501342d1-f635-437c-a42d-aa2de43b24bd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:44.653720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501342d1-f635-437c-a42d-aa2de43b24bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:44.666047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:44.734948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-501342d1-f635-437c-a42d-aa2de43b24bd None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:44.736747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-501342d1-f635-437c-a42d-aa2de43b24bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:44.866022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 688/1381] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:45.191245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d445dc-6544-4a5a-ba7a-a3c2e6f0c3c5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:45.193148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-73d445dc-6544-4a5a-ba7a-a3c2e6f0c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:45.384155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:45.399437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d445dc-6544-4a5a-ba7a-a3c2e6f0c3c5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:45.562493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:45.563197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:45.563475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:45.563746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:45.565423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:45.688009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:45.691097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ec65c-7f40-4e86-858b-1f8c4c15cf94 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:45.704719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:45.716145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 694/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:47.946393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6e0a5-d8bd-421e-9d62-8571c58fb264 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:47.946936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6e0a5-d8bd-421e-9d62-8571c58fb264 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:47.947262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6e0a5-d8bd-421e-9d62-8571c58fb264 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:47.949465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 689/1383] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:27:47.955134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048f7ed-e943-4a9b-985c-cd8ef37c9017 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:47.955915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048f7ed-e943-4a9b-985c-cd8ef37c9017 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:47.955915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048f7ed-e943-4a9b-985c-cd8ef37c9017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:48.038163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7048f7ed-e943-4a9b-985c-cd8ef37c9017 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:48.039881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7048f7ed-e943-4a9b-985c-cd8ef37c9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:48.169006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 695/1384] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:48.494224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6e0a5-d8bd-421e-9d62-8571c58fb264 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:48.496247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-53b6e0a5-d8bd-421e-9d62-8571c58fb264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:48.608497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:48.692585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6e0a5-d8bd-421e-9d62-8571c58fb264 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:48.840905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:48.841579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:48.841953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:48.842281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:48.844232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:48.915167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:48.968624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:48.970293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f0e83-3885-4161-be02-9360144071d8 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:48.992715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 690/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:51.009078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb2aee-5d25-46ef-80b3-19679542543e None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:51.009814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb2aee-5d25-46ef-80b3-19679542543e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:51.009814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb2aee-5d25-46ef-80b3-19679542543e None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:51.010906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 696/1386] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:51.016606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2d9e-5604-4813-a8e2-6e7b3a8b5512 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:51.017230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2d9e-5604-4813-a8e2-6e7b3a8b5512 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:51.017230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2d9e-5604-4813-a8e2-6e7b3a8b5512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:51.091682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d04e2d9e-5604-4813-a8e2-6e7b3a8b5512 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:51.092981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d04e2d9e-5604-4813-a8e2-6e7b3a8b5512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:51.221346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 691/1387] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:51.519540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb2aee-5d25-46ef-80b3-19679542543e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:51.522906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d7bb2aee-5d25-46ef-80b3-19679542543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:51.625337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:51.704909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb2aee-5d25-46ef-80b3-19679542543e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:51.833113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:51.833724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:51.834164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:51.834636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:51.836371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:51.879431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:51.936135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:51.937864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa5165c-e2e6-437d-ae8f-f8e7f1b4937e service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:51.957821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 697/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:27:53.915920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e222-62c3-482f-a8ba-05eaf445c10c None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:53.916799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e222-62c3-482f-a8ba-05eaf445c10c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:53.916799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e222-62c3-482f-a8ba-05eaf445c10c None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:53.918129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 692/1389] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:53.923841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d6e90-e076-4912-acd0-d1b20a51b769 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:53.924377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d6e90-e076-4912-acd0-d1b20a51b769 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:53.924497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d6e90-e076-4912-acd0-d1b20a51b769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:53.996631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bf6d6e90-e076-4912-acd0-d1b20a51b769 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:53.997853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bf6d6e90-e076-4912-acd0-d1b20a51b769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:54.082959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b23fb-0045-4d76-a21a-51820c47d90e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:54.083238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b23fb-0045-4d76-a21a-51820c47d90e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:54.083521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b23fb-0045-4d76-a21a-51820c47d90e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:54.085539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 693/1390] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:27:54.090747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1fa4-e733-4c22-80b3-6bd587684899 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:54.091039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1fa4-e733-4c22-80b3-6bd587684899 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:54.091311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1fa4-e733-4c22-80b3-6bd587684899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:54.103514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 698/1391] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:54.129075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:54.161823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-424f1fa4-e733-4c22-80b3-6bd587684899 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:54.163824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-424f1fa4-e733-4c22-80b3-6bd587684899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:54.267316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 694/1392] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:27:54.409096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ae91-224f-4c80-99b2-0aa4d14549a8 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:54.409383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ae91-224f-4c80-99b2-0aa4d14549a8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:54.409659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ae91-224f-4c80-99b2-0aa4d14549a8 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:54.411002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 699/1393] 10.222.0.65 () {54 vars in 967 bytes} [Wed Aug 31 00:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:54.417340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:54.417815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:54.418245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:54.498363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] MFA Rules not processed for user `ad5664c09ed44105a1083cc90b3b739a`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:54.501674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:54.555273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:54.563015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109ae91-224f-4c80-99b2-0aa4d14549a8 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:54.564562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a109ae91-224f-4c80-99b2-0aa4d14549a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:54.580766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:54.587200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 695/1394] 10.222.0.65 () {58 vars in 1064 bytes} [Wed Aug 31 00:27:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:54.594653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:54.603636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ae51f-92d4-4e6b-ac82-ad4941737dcb None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:54.682337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:54.682928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:54.683207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:54.683495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:54.685184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:54.723472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109ae91-224f-4c80-99b2-0aa4d14549a8 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:54.800550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:54.802115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b9ff3-626d-47e6-bece-d647c8d92964 service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:54.815522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:54.816358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:54.821664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 696/1395] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:27:54.843773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:54.844425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:54.844706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:54.844977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:54.846619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:54.942108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:54.944340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41ba34-c995-4821-b290-892d8b343058 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:54.964524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 700/1396] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:27:55.746650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec4440-3017-4bb7-8ddc-3b3362c83c7d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:55.747322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec4440-3017-4bb7-8ddc-3b3362c83c7d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:55.747516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec4440-3017-4bb7-8ddc-3b3362c83c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:55.818722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a8ec4440-3017-4bb7-8ddc-3b3362c83c7d None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:55.820028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a8ec4440-3017-4bb7-8ddc-3b3362c83c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:55.907928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 697/1397] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:27:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:55.953201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bf551-984e-4a9a-9392-aeda82f6a7f5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:55.953693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bf551-984e-4a9a-9392-aeda82f6a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:55.953873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bf551-984e-4a9a-9392-aeda82f6a7f5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:55.955112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 701/1398] 10.222.0.65 () {54 vars in 943 bytes} [Wed Aug 31 00:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:55.961565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b4374-2226-4fa2-b8b3-8a9ce9fa93c4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:55.961857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b4374-2226-4fa2-b8b3-8a9ce9fa93c4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:55.962132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b4374-2226-4fa2-b8b3-8a9ce9fa93c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:56.042051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-403b4374-2226-4fa2-b8b3-8a9ce9fa93c4 None None] MFA Rules not processed for user `1468fb64bc6d4b1899be323c0116c8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:56.043199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-403b4374-2226-4fa2-b8b3-8a9ce9fa93c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:56.072706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:56.154593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 698/1399] 10.222.0.65 () {58 vars in 1040 bytes} [Wed Aug 31 00:27:55 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:56.164715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bf551-984e-4a9a-9392-aeda82f6a7f5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:56.166526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-988bf551-984e-4a9a-9392-aeda82f6a7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:56.273656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:56.314804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bf551-984e-4a9a-9392-aeda82f6a7f5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:56.423675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:56.424288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:56.424606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:56.424867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:56.426499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:56.518485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:56.520145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7b44a-ac91-4acf-9a66-c38035fb6f83 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:56.525291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:56.536364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 702/1400] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:27:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:27:57.078003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13d45-0fdb-482e-ae66-218cf0c4e2a8 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:57.078526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13d45-0fdb-482e-ae66-218cf0c4e2a8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:57.078526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13d45-0fdb-482e-ae66-218cf0c4e2a8 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:57.079942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 699/1401] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:27:57.086260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e091cd3-3843-4a83-a80b-4cecbc76fe86 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:57.087081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e091cd3-3843-4a83-a80b-4cecbc76fe86 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:57.087536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e091cd3-3843-4a83-a80b-4cecbc76fe86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:57.175486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1e091cd3-3843-4a83-a80b-4cecbc76fe86 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:27:57.176962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1e091cd3-3843-4a83-a80b-4cecbc76fe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:57.301756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 703/1402] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:27:57.616608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13d45-0fdb-482e-ae66-218cf0c4e2a8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:27:57.618290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-23b13d45-0fdb-482e-ae66-218cf0c4e2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:57.806615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13d45-0fdb-482e-ae66-218cf0c4e2a8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:27:57.810360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:27:57.948920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:27:57.949533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:27:57.949818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:27:57.950071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:27:57.951697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:27:58.062037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:27:58.063594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a96f0-a3d7-4917-b06f-d6ef6f7da66f service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:27:58.085524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 700/1403] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:00.244237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bd43b-1d25-42ee-9f63-58c0d5a708b9 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:00.244932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bd43b-1d25-42ee-9f63-58c0d5a708b9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:00.244932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bd43b-1d25-42ee-9f63-58c0d5a708b9 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:00.246152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 704/1404] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:00.251864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e761e-206f-4270-afa8-190f6d0bc83e None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:00.252419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e761e-206f-4270-afa8-190f6d0bc83e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:00.252419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e761e-206f-4270-afa8-190f6d0bc83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:00.263192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:00.329566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-610e761e-206f-4270-afa8-190f6d0bc83e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:00.330797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-610e761e-206f-4270-afa8-190f6d0bc83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:00.439164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 701/1405] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:00.770917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bd43b-1d25-42ee-9f63-58c0d5a708b9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:00.772581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0f3bd43b-1d25-42ee-9f63-58c0d5a708b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:00.864186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:00.951028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bd43b-1d25-42ee-9f63-58c0d5a708b9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:01.097560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:01.098083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:01.098356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:01.098618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:01.100460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:01.147386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:01.203839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:01.205543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36066106-f095-44f6-a690-a98e18c82910 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:01.227804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 705/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:03.343916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd0814-faec-4069-9889-aedb46fcd00e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:03.344503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd0814-faec-4069-9889-aedb46fcd00e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:03.344503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd0814-faec-4069-9889-aedb46fcd00e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:03.346562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 702/1407] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:03.352456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75bc5d-339f-477f-af49-ae36f4217bb3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:03.353031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75bc5d-339f-477f-af49-ae36f4217bb3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:03.353176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75bc5d-339f-477f-af49-ae36f4217bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:03.438639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ea75bc5d-339f-477f-af49-ae36f4217bb3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:03.439820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ea75bc5d-339f-477f-af49-ae36f4217bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:03.573410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 706/1408] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:03.875695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd0814-faec-4069-9889-aedb46fcd00e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:03.877437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fbcd0814-faec-4069-9889-aedb46fcd00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:03.975875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:04.047776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd0814-faec-4069-9889-aedb46fcd00e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:04.179802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:04.180551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:04.180839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:04.181110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:04.182771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:04.242070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:04.294318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:04.296225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763d00-a2e0-499d-ab19-16d1f32d5301 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:04.316433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 703/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:13.570989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=93 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:14.313067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=24 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:20.434548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41baab4-443f-40e2-8f55-de28f9322813 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:20.435081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41baab4-443f-40e2-8f55-de28f9322813 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:20.435081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41baab4-443f-40e2-8f55-de28f9322813 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:20.436362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 707/1410] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 31 00:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:20.441684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d4549-e260-4319-a06b-2c069f191807 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:20.442773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d4549-e260-4319-a06b-2c069f191807 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:20.443062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d4549-e260-4319-a06b-2c069f191807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:20.525650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e00d4549-e260-4319-a06b-2c069f191807 None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:20.526919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e00d4549-e260-4319-a06b-2c069f191807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:20.650329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 704/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 31 00:28:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:20.666468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9331b-974d-4c7d-9c4e-69e418886d75 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:20.666743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9331b-974d-4c7d-9c4e-69e418886d75 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:20.667007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9331b-974d-4c7d-9c4e-69e418886d75 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:20.668438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 708/1412] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 31 00:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:20.673316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188bfed-294c-4782-8992-40b5a4caac47 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:20.673602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188bfed-294c-4782-8992-40b5a4caac47 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:20.673873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188bfed-294c-4782-8992-40b5a4caac47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:20.755004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:20.758572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d188bfed-294c-4782-8992-40b5a4caac47 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:20.759804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d188bfed-294c-4782-8992-40b5a4caac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:20.888226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 705/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 31 00:28:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:23.343390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf3081-51bc-4702-9084-aa9047ca7bca None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:23.343963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf3081-51bc-4702-9084-aa9047ca7bca None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:23.343963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf3081-51bc-4702-9084-aa9047ca7bca None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:23.345305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:23.350693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b011e-b777-45b3-8740-0c52607ae708 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:23.351127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b011e-b777-45b3-8740-0c52607ae708 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:23.351250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b011e-b777-45b3-8740-0c52607ae708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:23.420188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7d7b011e-b777-45b3-8740-0c52607ae708 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:23.421411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7d7b011e-b777-45b3-8740-0c52607ae708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:23.478451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:23.523749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:23.819861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf3081-51bc-4702-9084-aa9047ca7bca None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:23.821633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-73cf3081-51bc-4702-9084-aa9047ca7bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:23.992148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf3081-51bc-4702-9084-aa9047ca7bca None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:24.075998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:24.120193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:24.120834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:24.121110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:24.121369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:24.122981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:24.227885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:24.229565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603c672-6558-4c77-9ad4-6e69e7cf2408 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:24.249884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 710/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:28:25.171965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70bfe0-4106-4f7d-9d42-f391a81c681a None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:25.172614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70bfe0-4106-4f7d-9d42-f391a81c681a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:25.172614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70bfe0-4106-4f7d-9d42-f391a81c681a None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:25.173867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:25.180567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fac11-03f3-46ed-ac4c-fd45f7df4bd6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:25.181170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fac11-03f3-46ed-ac4c-fd45f7df4bd6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:25.181170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fac11-03f3-46ed-ac4c-fd45f7df4bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:25.265693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2b9fac11-03f3-46ed-ac4c-fd45f7df4bd6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:25.266895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2b9fac11-03f3-46ed-ac4c-fd45f7df4bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:25.297317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:25.374658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:28:25.528047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70bfe0-4106-4f7d-9d42-f391a81c681a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:25.529605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ef70bfe0-4106-4f7d-9d42-f391a81c681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:25.677041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70bfe0-4106-4f7d-9d42-f391a81c681a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:25.720926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:25.795509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:25.796107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:25.796410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:25.796679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:25.798273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:25.889184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:25.890865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582385f8-26df-482a-9ae5-b3c5838dd179 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:25.909336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 708/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:26.813413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02155886-8874-4ff5-b1a2-28a954361e4a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:26.813959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02155886-8874-4ff5-b1a2-28a954361e4a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:26.814086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02155886-8874-4ff5-b1a2-28a954361e4a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:26.816060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:26.821409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5fb01-d8c7-4631-9125-b05a590b0c40 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:26.821894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5fb01-d8c7-4631-9125-b05a590b0c40 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:26.821990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5fb01-d8c7-4631-9125-b05a590b0c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:26.880700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:26.907624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d2f5fb01-d8c7-4631-9125-b05a590b0c40 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:26.908900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d2f5fb01-d8c7-4631-9125-b05a590b0c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:27.012744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:27.166295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02155886-8874-4ff5-b1a2-28a954361e4a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:27.167941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-02155886-8874-4ff5-b1a2-28a954361e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:27.326172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02155886-8874-4ff5-b1a2-28a954361e4a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:27.339764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:27.451055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:27.451572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:27.451844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:27.452112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:27.453786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:27.555155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:27.556803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e99561-ac9a-418c-99e5-22a9cef61334 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:27.575512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 713/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:28.604569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cf3ba-ff44-4961-907d-f38479dd481b None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:28.605301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cf3ba-ff44-4961-907d-f38479dd481b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:28.605301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cf3ba-ff44-4961-907d-f38479dd481b None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:28.606647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:28.613725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077d61-f11a-4a15-ae1f-a6fea2766303 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:28.614487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077d61-f11a-4a15-ae1f-a6fea2766303 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:28.614487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09077d61-f11a-4a15-ae1f-a6fea2766303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:28.646484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:28.732369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-09077d61-f11a-4a15-ae1f-a6fea2766303 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:28.733753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-09077d61-f11a-4a15-ae1f-a6fea2766303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:28.864393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:29.034155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cf3ba-ff44-4961-907d-f38479dd481b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:29.035870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-710cf3ba-ff44-4961-907d-f38479dd481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:29.174625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:29.225075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cf3ba-ff44-4961-907d-f38479dd481b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:29.374589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:29.375391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:29.375669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:29.375942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:29.377758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:29.480212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:29.496314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:29.499096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc861fc3-4bc2-425d-8d7c-85ffc893a790 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:29.519482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 711/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:30.451049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c041a7b-2779-4f13-bcb4-e56e75da5ae6 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:30.451749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c041a7b-2779-4f13-bcb4-e56e75da5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:30.451749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c041a7b-2779-4f13-bcb4-e56e75da5ae6 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:30.452975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:30.459432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339571c3-415f-447d-b18b-9d512b3553b4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:30.459930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339571c3-415f-447d-b18b-9d512b3553b4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:30.460434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339571c3-415f-447d-b18b-9d512b3553b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:30.557301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-339571c3-415f-447d-b18b-9d512b3553b4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:30.558546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-339571c3-415f-447d-b18b-9d512b3553b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:30.675858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:30.832042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c041a7b-2779-4f13-bcb4-e56e75da5ae6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:30.833770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9c041a7b-2779-4f13-bcb4-e56e75da5ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:30.940030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:31.022110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c041a7b-2779-4f13-bcb4-e56e75da5ae6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:31.155851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:31.156537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:31.156813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:31.157079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:31.158752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:31.221686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:31.272528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:31.274264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af266f6a-4c60-4529-ae31-9f8f9327aea2 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:31.295995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 716/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:28:32.410407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d68ba-f090-4256-a5e1-56325604ba5e None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:32.411056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d68ba-f090-4256-a5e1-56325604ba5e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:32.411056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d68ba-f090-4256-a5e1-56325604ba5e None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:32.412604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:32.418401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9d83-96e7-489e-aca6-b872629543b9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:32.418927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9d83-96e7-489e-aca6-b872629543b9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:32.419031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9d83-96e7-489e-aca6-b872629543b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:32.508008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5fcb9d83-96e7-489e-aca6-b872629543b9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:32.509267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5fcb9d83-96e7-489e-aca6-b872629543b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:32.616994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:32.772688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d68ba-f090-4256-a5e1-56325604ba5e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:32.774286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-860d68ba-f090-4256-a5e1-56325604ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:32.828539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:32.942990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d68ba-f090-4256-a5e1-56325604ba5e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:33.087139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:33.087654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:33.087936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:33.088206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:33.089904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:33.098482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:33.194606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:33.196288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e5e29-1fe0-4030-8325-4a24ba418a41 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:33.218034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 714/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:34.271616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b2d1a-6689-4e24-afc7-2c45691848cb None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:34.272190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b2d1a-6689-4e24-afc7-2c45691848cb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:34.272358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b2d1a-6689-4e24-afc7-2c45691848cb None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:34.275150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:34.280647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdccbc-6fb8-4a92-9e2e-32fb18716e69 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:34.281282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdccbc-6fb8-4a92-9e2e-32fb18716e69 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:34.281282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdccbc-6fb8-4a92-9e2e-32fb18716e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:34.368047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-28bdccbc-6fb8-4a92-9e2e-32fb18716e69 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:34.369360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-28bdccbc-6fb8-4a92-9e2e-32fb18716e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:34.443624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:34.480125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:34.641096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b2d1a-6689-4e24-afc7-2c45691848cb None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:34.643438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7d3b2d1a-6689-4e24-afc7-2c45691848cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:34.666815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:34.813081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b2d1a-6689-4e24-afc7-2c45691848cb None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:34.909043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:34.942258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:34.942940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:34.943235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:34.943523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:34.945457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:35.053521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:35.055138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fffcb1-819c-4d72-99da-f1f7789c2668 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:35.073791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 719/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:36.003548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4fc1b-58a5-4a23-97ac-a1c332c935a4 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:36.004222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4fc1b-58a5-4a23-97ac-a1c332c935a4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:36.004448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4fc1b-58a5-4a23-97ac-a1c332c935a4 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:36.005839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:28:36.011569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e2f5-6baf-4bd6-8197-20d9b13d37fe None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:36.012037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e2f5-6baf-4bd6-8197-20d9b13d37fe None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:36.012144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e2f5-6baf-4bd6-8197-20d9b13d37fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:36.092818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1cc3e2f5-6baf-4bd6-8197-20d9b13d37fe None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:36.094036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1cc3e2f5-6baf-4bd6-8197-20d9b13d37fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:36.137800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:36.208047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:36.366976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4fc1b-58a5-4a23-97ac-a1c332c935a4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:36.368464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cce4fc1b-58a5-4a23-97ac-a1c332c935a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:36.525954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4fc1b-58a5-4a23-97ac-a1c332c935a4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:36.593754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:36.655895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:36.656450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:36.656724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:36.656989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:36.658646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:36.766365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:36.768866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54337beb-2e4f-45bc-9108-b1a49c6bf2f4 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:36.791748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 717/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:28:37.713384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b075-a3d5-46a6-8915-1cb51b1cfb19 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:37.713961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b075-a3d5-46a6-8915-1cb51b1cfb19 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:37.713961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b075-a3d5-46a6-8915-1cb51b1cfb19 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:37.715278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:37.720234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798c9ff-0183-4c43-a16f-6fe286fe7f53 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:37.720672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798c9ff-0183-4c43-a16f-6fe286fe7f53 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:37.720806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798c9ff-0183-4c43-a16f-6fe286fe7f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:37.799859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1798c9ff-0183-4c43-a16f-6fe286fe7f53 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:37.801206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1798c9ff-0183-4c43-a16f-6fe286fe7f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:37.803280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:37.901791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:38.052465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178b075-a3d5-46a6-8915-1cb51b1cfb19 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:38.053925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a178b075-a3d5-46a6-8915-1cb51b1cfb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:38.219276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178b075-a3d5-46a6-8915-1cb51b1cfb19 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:38.250133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:38.348593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:38.349112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:38.349385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:38.349647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:38.351269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:38.447023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:38.448717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bbca0-a762-4a14-9b12-3ffc97bbf7a2 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:38.468166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 722/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:40.428583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c0e05-d28f-41f2-93b7-0819002c16ba None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:40.429132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c0e05-d28f-41f2-93b7-0819002c16ba None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:40.429132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c0e05-d28f-41f2-93b7-0819002c16ba None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:40.430513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:40.436993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4e45-d576-49ce-95b3-ea0a16b77c50 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:40.437518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4e45-d576-49ce-95b3-ea0a16b77c50 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:40.437661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4e45-d576-49ce-95b3-ea0a16b77c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:40.484391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:40.510810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-144a4e45-d576-49ce-95b3-ea0a16b77c50 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:40.512041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-144a4e45-d576-49ce-95b3-ea0a16b77c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:40.630350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:40.918823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c0e05-d28f-41f2-93b7-0819002c16ba None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:40.920411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-314c0e05-d28f-41f2-93b7-0819002c16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:41.032396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c0e05-d28f-41f2-93b7-0819002c16ba None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:41.067457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:41.113536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:41.116761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:41.116761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:41.116761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:41.118428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:41.243295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:41.244804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad38d4-668a-40c8-be92-80cd26a5b1ae service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:41.265755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 720/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:43.418149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef113c9-acae-4606-9796-d9bd0f648978 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:43.418673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef113c9-acae-4606-9796-d9bd0f648978 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:43.418673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef113c9-acae-4606-9796-d9bd0f648978 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:43.419975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:43.426335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4153e-6e64-4bfb-af4d-69199abf9c89 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:43.426793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4153e-6e64-4bfb-af4d-69199abf9c89 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:43.426889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4153e-6e64-4bfb-af4d-69199abf9c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:43.496982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a8b4153e-6e64-4bfb-af4d-69199abf9c89 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:43.498215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a8b4153e-6e64-4bfb-af4d-69199abf9c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:43.511862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:43.598479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:43.884081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef113c9-acae-4606-9796-d9bd0f648978 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:43.885652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1ef113c9-acae-4606-9796-d9bd0f648978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:43.990435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef113c9-acae-4606-9796-d9bd0f648978 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:44.002607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:44.066319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:44.066919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:44.067193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:44.067461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:44.069108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:44.182557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:44.183898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844e6de-bd2a-43f3-8036-b1289137cad7 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:44.202325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 725/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:46.582217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b9f27-0655-48a1-8061-550016c53584 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:46.582930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b9f27-0655-48a1-8061-550016c53584 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:46.582930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b9f27-0655-48a1-8061-550016c53584 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:46.584163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:46.589580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc88c0-4e47-47f2-a0a1-8524f495b66f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:46.589993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc88c0-4e47-47f2-a0a1-8524f495b66f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:46.590129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc88c0-4e47-47f2-a0a1-8524f495b66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:46.639453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:46.657947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-29bc88c0-4e47-47f2-a0a1-8524f495b66f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:46.659749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-29bc88c0-4e47-47f2-a0a1-8524f495b66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:46.760350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:47.049362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b9f27-0655-48a1-8061-550016c53584 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:47.051167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b01b9f27-0655-48a1-8061-550016c53584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:47.162791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b9f27-0655-48a1-8061-550016c53584 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:47.206458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:47.241507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:47.242323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:47.242799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:47.243381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:47.245049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:47.357257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:47.358628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc523271-800e-4b95-88b4-ca47876b20ce service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:47.377270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 723/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:49.490423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfb390-4998-4324-8586-a9c461f6c5a7 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:49.490971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfb390-4998-4324-8586-a9c461f6c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:49.491136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfb390-4998-4324-8586-a9c461f6c5a7 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:49.492875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:49.498536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddc94c-8f2f-465f-b6c4-2ee119fe39b9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:49.499143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddc94c-8f2f-465f-b6c4-2ee119fe39b9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:49.499143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddc94c-8f2f-465f-b6c4-2ee119fe39b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:49.579342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0eddc94c-8f2f-465f-b6c4-2ee119fe39b9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:49.580708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0eddc94c-8f2f-465f-b6c4-2ee119fe39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:49.602553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:49.688421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:49.986921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfb390-4998-4324-8586-a9c461f6c5a7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:49.989516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-55bfb390-4998-4324-8586-a9c461f6c5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:50.103271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfb390-4998-4324-8586-a9c461f6c5a7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:50.124794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:50.189785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:50.191122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:50.191408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:50.191686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:50.193418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:50.326153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:50.327604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f605cb-9580-4cb5-8764-4dc2e41928c2 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:50.350089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 728/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:52.456578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373327d0-4833-4d28-ab0a-2cec36251e80 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:52.457157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373327d0-4833-4d28-ab0a-2cec36251e80 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:52.457157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373327d0-4833-4d28-ab0a-2cec36251e80 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:52.458916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:52.464020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb111a4-26cd-42a8-864f-1d2c1ee231ef None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:52.464465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb111a4-26cd-42a8-864f-1d2c1ee231ef None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:52.464595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb111a4-26cd-42a8-864f-1d2c1ee231ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:52.534045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:52.537509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0fb111a4-26cd-42a8-864f-1d2c1ee231ef None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:52.538787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0fb111a4-26cd-42a8-864f-1d2c1ee231ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:52.640009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:52.924071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373327d0-4833-4d28-ab0a-2cec36251e80 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:52.925793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-373327d0-4833-4d28-ab0a-2cec36251e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:53.042783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373327d0-4833-4d28-ab0a-2cec36251e80 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:53.097298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:53.126192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:53.126192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:53.126192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:53.126658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:53.128677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:53.283932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:53.286667 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83fb69c-aa44-4a2a-878e-8299392666f2 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:53.312599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 726/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:55.355576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836abbb3-d7ee-4f34-bc24-1a038b84a52a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:55.356146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836abbb3-d7ee-4f34-bc24-1a038b84a52a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:55.356278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836abbb3-d7ee-4f34-bc24-1a038b84a52a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:55.357806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:55.364408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da681-802e-42ca-b7b5-4f26711dee45 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:55.364906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da681-802e-42ca-b7b5-4f26711dee45 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:55.364999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da681-802e-42ca-b7b5-4f26711dee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:55.434479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-469da681-802e-42ca-b7b5-4f26711dee45 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:55.435643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-469da681-802e-42ca-b7b5-4f26711dee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:55.464864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:55.537181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:55.825570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836abbb3-d7ee-4f34-bc24-1a038b84a52a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:55.827123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-836abbb3-d7ee-4f34-bc24-1a038b84a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:55.935990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836abbb3-d7ee-4f34-bc24-1a038b84a52a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:55.964378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:56.011455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:56.012062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:56.012360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:56.012633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:56.014255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:56.126174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:56.127544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7efacb-d001-4308-84a2-1c1e6f56bf2f service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:56.146121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 731/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:58.148799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41f0c-2e82-4653-bb6e-4287366bbcb6 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:58.149377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41f0c-2e82-4653-bb6e-4287366bbcb6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:58.149472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41f0c-2e82-4653-bb6e-4287366bbcb6 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:58.150981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:28:58.157954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a81a98-989c-445d-8fc3-a98e892d2f6a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:58.157954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a81a98-989c-445d-8fc3-a98e892d2f6a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:58.159303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a81a98-989c-445d-8fc3-a98e892d2f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:58.232618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-90a81a98-989c-445d-8fc3-a98e892d2f6a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:58.233846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-90a81a98-989c-445d-8fc3-a98e892d2f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:58.244605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:58.346578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:58.633728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41f0c-2e82-4653-bb6e-4287366bbcb6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:58.635299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c6e41f0c-2e82-4653-bb6e-4287366bbcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:58.745995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41f0c-2e82-4653-bb6e-4287366bbcb6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:58.805679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:58.826615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:58.827196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:58.827468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:58.827730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:58.829514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:58.952585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:58.954045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b18bdc-0256-4211-8820-51771e1f246a service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:58.973588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 729/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:00.958615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663fec-fc58-4906-8e03-9f1143d1be52 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:00.959293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663fec-fc58-4906-8e03-9f1143d1be52 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:00.959293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663fec-fc58-4906-8e03-9f1143d1be52 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:00.960518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:00.965837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578a5f6-b2e3-41ca-8eba-f0eead36d037 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:00.966253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578a5f6-b2e3-41ca-8eba-f0eead36d037 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:00.966402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578a5f6-b2e3-41ca-8eba-f0eead36d037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:01.036917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9578a5f6-b2e3-41ca-8eba-f0eead36d037 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:01.038290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9578a5f6-b2e3-41ca-8eba-f0eead36d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:01.077786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:01.137978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:01.424806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663fec-fc58-4906-8e03-9f1143d1be52 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:01.426433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ca663fec-fc58-4906-8e03-9f1143d1be52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:01.536559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663fec-fc58-4906-8e03-9f1143d1be52 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:01.580085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:01.620088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:01.620808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:01.621139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:01.621408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:01.623041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:01.745289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:01.746784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278dd82-1e07-4fa8-8463-b7138f425d5d service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:01.769444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 734/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:03.851541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e78e1-f23b-470e-b2f0-750e47ca9fd2 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:03.852146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e78e1-f23b-470e-b2f0-750e47ca9fd2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:03.852146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e78e1-f23b-470e-b2f0-750e47ca9fd2 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:03.853474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:03.858824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178b14f-9d4c-479d-9623-449c09c2b0b8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:03.859239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178b14f-9d4c-479d-9623-449c09c2b0b8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:03.859366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178b14f-9d4c-479d-9623-449c09c2b0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:03.936599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0178b14f-9d4c-479d-9623-449c09c2b0b8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:03.937811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0178b14f-9d4c-479d-9623-449c09c2b0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:03.956583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:04.037292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:04.320486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e78e1-f23b-470e-b2f0-750e47ca9fd2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:04.322049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-832e78e1-f23b-470e-b2f0-750e47ca9fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:04.440750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e78e1-f23b-470e-b2f0-750e47ca9fd2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:04.518046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:04.526818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:04.527432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:04.527708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:04.527972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:04.529597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:04.653629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:04.654994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5378ab5-6cc8-4daf-9d69-67359f69d899 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:04.674810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 732/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:06.609911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ca48-822e-4cd4-a9f1-050b8cd10601 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:06.610742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ca48-822e-4cd4-a9f1-050b8cd10601 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:06.610742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ca48-822e-4cd4-a9f1-050b8cd10601 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:06.611886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:06.617262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8583179-8dd0-4ccf-ad7f-a860f1f54f45 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:06.617839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8583179-8dd0-4ccf-ad7f-a860f1f54f45 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:06.618122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8583179-8dd0-4ccf-ad7f-a860f1f54f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:06.687050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a8583179-8dd0-4ccf-ad7f-a860f1f54f45 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:06.688430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a8583179-8dd0-4ccf-ad7f-a860f1f54f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:06.738436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:06.795681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:07.080183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ca48-822e-4cd4-a9f1-050b8cd10601 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:07.081904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-32e0ca48-822e-4cd4-a9f1-050b8cd10601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:07.307268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ca48-822e-4cd4-a9f1-050b8cd10601 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:07.357439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:07.383353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:07.387625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:07.387625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:07.387625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:07.387625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:07.504538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:07.505979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e883dd-6beb-4a94-925a-868713df8b00 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:07.525606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 737/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:29:16.793860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=22 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:17.522923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=66 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:28.283389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a269883-cfe8-491c-a4f2-3cbb913de3c5 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:28.283943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a269883-cfe8-491c-a4f2-3cbb913de3c5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:28.283943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a269883-cfe8-491c-a4f2-3cbb913de3c5 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:28.285351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 734/1471] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:28.290811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ed2ce-04ab-4ec1-8427-65f2cbb6b008 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:28.291257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ed2ce-04ab-4ec1-8427-65f2cbb6b008 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:28.291359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ed2ce-04ab-4ec1-8427-65f2cbb6b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:28.362610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5c0ed2ce-04ab-4ec1-8427-65f2cbb6b008 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:28.363834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5c0ed2ce-04ab-4ec1-8427-65f2cbb6b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:28.462805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 738/1472] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:28.766323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b02d9-22f0-4a08-98e5-0dd3c358e9c2 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:28.767182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b02d9-22f0-4a08-98e5-0dd3c358e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:28.767505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b02d9-22f0-4a08-98e5-0dd3c358e9c2 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:28.768967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 735/1473] 10.222.0.65 () {54 vars in 967 bytes} [Wed Aug 31 00:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:28.773923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffae96-aa5d-4e58-94c1-519656026a6a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:28.774208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffae96-aa5d-4e58-94c1-519656026a6a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:28.774478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffae96-aa5d-4e58-94c1-519656026a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:28.854420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-b5ffae96-aa5d-4e58-94c1-519656026a6a None None] MFA Rules not processed for user `ad5664c09ed44105a1083cc90b3b739a`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:28.855624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b5ffae96-aa5d-4e58-94c1-519656026a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:28.877629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:28.940943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 739/1474] 10.222.0.65 () {58 vars in 1064 bytes} [Wed Aug 31 00:29:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:29:28.948533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b02d9-22f0-4a08-98e5-0dd3c358e9c2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:28.950088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4f5b02d9-22f0-4a08-98e5-0dd3c358e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:29.060151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b02d9-22f0-4a08-98e5-0dd3c358e9c2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:29.138771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:29.139384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:29.139689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:29.140192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:29.142244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:29.201773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:29.259324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:29.260827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a20628-bbeb-4d57-a7f1-29775ed78095 service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:29.280618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 736/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:31.350235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef3dc1-a293-4174-869e-6bd53adec3fd None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:31.350805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef3dc1-a293-4174-869e-6bd53adec3fd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:31.350805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef3dc1-a293-4174-869e-6bd53adec3fd None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:31.352156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 740/1476] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:31.358022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df077f6a-1e25-403f-979e-2663c58dae67 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:31.358601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df077f6a-1e25-403f-979e-2663c58dae67 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:31.358601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df077f6a-1e25-403f-979e-2663c58dae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:31.438994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-df077f6a-1e25-403f-979e-2663c58dae67 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:31.439703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-df077f6a-1e25-403f-979e-2663c58dae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:31.552127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 737/1477] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:31.829516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef3dc1-a293-4174-869e-6bd53adec3fd None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:31.831083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8eef3dc1-a293-4174-869e-6bd53adec3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:31.980568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef3dc1-a293-4174-869e-6bd53adec3fd None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:32.040746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:32.091272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:32.091789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:32.092059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:32.092318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:32.092965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:32.095508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:29:32.095508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:29:32.095508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:29:32.096670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2e19d-9609-4116-8583-6447f24fee7c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:32.116947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 741/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 00:29:31 2022] GET /identity/v3/projects => generated 2553 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:29:34.097752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b7f4-3f6f-4c07-8125-5e97bf70d588 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:34.098380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b7f4-3f6f-4c07-8125-5e97bf70d588 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:34.098380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b7f4-3f6f-4c07-8125-5e97bf70d588 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:34.099648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 738/1479] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:34.104940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd6dbe-2b3a-433d-bad6-9bb0a1153640 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:34.105403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd6dbe-2b3a-433d-bad6-9bb0a1153640 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:34.105501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd6dbe-2b3a-433d-bad6-9bb0a1153640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:34.175500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-65cd6dbe-2b3a-433d-bad6-9bb0a1153640 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:34.176766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-65cd6dbe-2b3a-433d-bad6-9bb0a1153640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:34.277542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 742/1480] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:34.577537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572b7f4-3f6f-4c07-8125-5e97bf70d588 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:34.579514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f572b7f4-3f6f-4c07-8125-5e97bf70d588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:34.589677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:34.739134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572b7f4-3f6f-4c07-8125-5e97bf70d588 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:34.823050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:34.856579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:34.857186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:34.857469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:34.857737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:34.859360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:34.954862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:34.956521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785ca8a-8e84-4ec2-9efd-0901e6596f12 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:34.975149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 739/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:37.154887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8820e2-9320-46ea-af29-528e17884d5b None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:37.196994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8820e2-9320-46ea-af29-528e17884d5b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:37.196994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8820e2-9320-46ea-af29-528e17884d5b None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:37.196994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 743/1482] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:37.198208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ff10-3f3f-486d-b9d4-d0ca53ac2d20 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:37.198208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ff10-3f3f-486d-b9d4-d0ca53ac2d20 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:37.198208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ff10-3f3f-486d-b9d4-d0ca53ac2d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:37.237496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c544ff10-3f3f-486d-b9d4-d0ca53ac2d20 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:37.238770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c544ff10-3f3f-486d-b9d4-d0ca53ac2d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:37.285537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:37.339743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 740/1483] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:37.629596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8820e2-9320-46ea-af29-528e17884d5b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:37.631189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9e8820e2-9320-46ea-af29-528e17884d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:37.655324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:37.740886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8820e2-9320-46ea-af29-528e17884d5b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:37.817729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:37.818336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:37.818609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:37.818873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:37.821615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:37.898696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:37.936349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:37.938354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baa448-9157-4f59-984a-f65cc8ace4fa service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:37.957326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 744/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:39.975979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4319d-b320-48b1-9899-99be78078964 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:39.976728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4319d-b320-48b1-9899-99be78078964 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:39.976728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4319d-b320-48b1-9899-99be78078964 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:39.977972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 741/1485] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:39.983546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8935334-a12b-405c-a3f5-01a1935e3acd None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:39.984040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8935334-a12b-405c-a3f5-01a1935e3acd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:39.984155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8935334-a12b-405c-a3f5-01a1935e3acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:40.063325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a8935334-a12b-405c-a3f5-01a1935e3acd None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:40.065831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a8935334-a12b-405c-a3f5-01a1935e3acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:40.185489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 745/1486] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:40.483063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4319d-b320-48b1-9899-99be78078964 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:40.485291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7cd4319d-b320-48b1-9899-99be78078964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:40.604641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4319d-b320-48b1-9899-99be78078964 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:40.684434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:40.686301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:40.686914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:40.687192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:40.687563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:40.689798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:40.804509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:40.805956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7f8e0-6132-45e8-8fef-6b214a776ebe service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:40.824908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 742/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:42.916269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e961d5-263b-499f-b8d9-0504a0ecaa78 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:42.916869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e961d5-263b-499f-b8d9-0504a0ecaa78 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:42.916869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e961d5-263b-499f-b8d9-0504a0ecaa78 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:42.918191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 746/1488] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:42.923496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7464d91-6277-4cb9-9f3c-2dab80b2008f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:42.923919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7464d91-6277-4cb9-9f3c-2dab80b2008f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:42.924055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7464d91-6277-4cb9-9f3c-2dab80b2008f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:42.990327 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b7464d91-6277-4cb9-9f3c-2dab80b2008f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:42.991575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b7464d91-6277-4cb9-9f3c-2dab80b2008f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:43.044595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:43.089700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 743/1489] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:29:43.383511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e961d5-263b-499f-b8d9-0504a0ecaa78 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:43.385078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-86e961d5-263b-499f-b8d9-0504a0ecaa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:43.423549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:43.494729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e961d5-263b-499f-b8d9-0504a0ecaa78 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:43.573878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:43.574486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:43.574757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:43.575028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:43.576663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:43.659124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:43.689554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:43.690898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3af70-94a9-4820-b9fa-832139987bc9 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:43.708961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 747/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:46.174926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03503b6d-75f9-43f1-99b9-046dc2509d03 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:46.175507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03503b6d-75f9-43f1-99b9-046dc2509d03 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:46.175507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03503b6d-75f9-43f1-99b9-046dc2509d03 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:46.177431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 744/1491] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 00:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:29:46.185694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9c11-8182-43f9-a153-c04f3b686f46 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:46.186225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9c11-8182-43f9-a153-c04f3b686f46 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:46.186685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9c11-8182-43f9-a153-c04f3b686f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:46.258632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-847c9c11-8182-43f9-a153-c04f3b686f46 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:29:46.259878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-847c9c11-8182-43f9-a153-c04f3b686f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:46.368801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 748/1492] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 00:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:29:46.680864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03503b6d-75f9-43f1-99b9-046dc2509d03 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:46.682541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-03503b6d-75f9-43f1-99b9-046dc2509d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:46.795486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03503b6d-75f9-43f1-99b9-046dc2509d03 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:46.878488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:46.879100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:46.879373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:46.879641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:46.881321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:46.891867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:47.002404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:47.003874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b2b2b-d22c-438a-aef4-eb6c807e2b01 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:47.023825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 745/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:29:56.366888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=82 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:57.021236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=50 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:12.367344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:12.368229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:12.368229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:12.457383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:31:12.458392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:12.571733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 749/1494] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:31:12.580712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:12.581999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:12.581999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:12.676369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:31:12.677533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:12.788400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 746/1495] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:31:12.793107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:12.794320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:12.805906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc4cf-aa0b-4369-8151-e6df92d6ae08 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:12.875162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:12.910501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:12.911119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:12.911394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:12.911665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:12.912314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:31:12.912645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:12.913704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:12.913704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:31:12.913704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:31:12.914600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:12.928371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 750/1496] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:31:12.936189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:12.950880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d4dc-2fbb-43b7-900c-18998880490a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:13.029228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:13.068703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:13.069340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:13.069624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:13.069925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:13.070568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:13.072476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:13.072476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:31:13.072476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:31:13.073694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:13.201608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 747/1497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:31:12 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:31:13.206393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:13.208061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:13.244069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b8cb7-a32f-4b12-b5e4-9bf2979bf3e5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:13.344027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:13.344668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:13.344949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:13.345220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:13.346595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:13.347657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:13.347657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:31:13.347657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:31:13.348899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:13.472360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 751/1498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:31:13 2022] POST /identity/v3/users => generated 290 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:31:13.476646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:13.479287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:13.665632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3b44e-2fb9-40c1-a0a3-0c2f073d0de9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:13.749668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:13.761010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:31:13.765133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:31:13.765748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:13.778245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 748/1499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:31:13.782315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:13.784179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:13.795881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1116ba1-fed4-49dd-8c06-f4bb4834fab8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:13.821570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:13.895366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:13.895966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:13.896233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:13.896521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:13.897149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:13.898157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:13.898157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:31:13.898157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:31:13.899026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1b8e4-3d16-4d63-8113-bd424f5fd437 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:13.912240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 752/1500] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:31:13.916619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:13.918139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:13.953730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f667a3-3e60-4b0f-ae79-d47b0a0ed566 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:14.056839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:14.057418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:14.057705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:14.057974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] PATH_INFO: `/v3/projects/89d80f89d138415d9129b0a3d33c0977/users/f0491cb2ef744edc8cfde96a2801560d/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:14.082023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d80f89d138415d9129b0a3d33c0977, user_id=f0491cb2ef744edc8cfde96a2801560d, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:14.083069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:14.083069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:31:14.083069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:31:14.085038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c1acf-8e7f-42d7-bff3-56b58f11554b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:14.205410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 749/1501] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:31:13 2022] PUT /identity/v3/projects/89d80f89d138415d9129b0a3d33c0977/users/f0491cb2ef744edc8cfde96a2801560d/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:31:14.217745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918690c2-197c-48d1-b22c-8c0d27a09f2b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:14.218017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918690c2-197c-48d1-b22c-8c0d27a09f2b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:14.218293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918690c2-197c-48d1-b22c-8c0d27a09f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:14.291931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-918690c2-197c-48d1-b22c-8c0d27a09f2b None None] MFA Rules not processed for user `f0491cb2ef744edc8cfde96a2801560d`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:31:14.293202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-918690c2-197c-48d1-b22c-8c0d27a09f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:14.374698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 753/1502] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:31:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:31:14.388682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963891bd-add2-4bd8-8470-9d0c44211345 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:14.388964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963891bd-add2-4bd8-8470-9d0c44211345 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:14.389232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963891bd-add2-4bd8-8470-9d0c44211345 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:14.390582 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 750/1503] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 31 00:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:31:14.396559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af0e83-f303-4777-abdb-431f26b4692c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:14.396847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af0e83-f303-4777-abdb-431f26b4692c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:14.397125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af0e83-f303-4777-abdb-431f26b4692c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:14.430036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:14.476852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e4af0e83-f303-4777-abdb-431f26b4692c None None] MFA Rules not processed for user `a765c1f74578435d8f29fbf959e5f087`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:31:14.478086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e4af0e83-f303-4777-abdb-431f26b4692c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:14.555643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 754/1504] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 31 00:31:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:31:14.563594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963891bd-add2-4bd8-8470-9d0c44211345 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:14.565897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-963891bd-add2-4bd8-8470-9d0c44211345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:14.616986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:14.672661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963891bd-add2-4bd8-8470-9d0c44211345 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:14.750002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:14.750614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:14.750890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:14.751161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:14.753668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:14.852150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:14.853639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:14.857237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:14.872111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 751/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:31:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:31:14.918230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e00a1-17d9-4ff3-9965-1308f6ec0d7f None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:14.918511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e00a1-17d9-4ff3-9965-1308f6ec0d7f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:14.918780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e00a1-17d9-4ff3-9965-1308f6ec0d7f None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:14.920054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 755/1506] 10.222.0.65 () {52 vars in 830 bytes} [Wed Aug 31 00:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:31:14.924433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:14.925819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:15.026843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ffebd-6aee-4f8a-a17a-611f5af15e6b service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:15.129226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:15.129856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:15.130131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:15.130400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] PATH_INFO: `/v3/users/f0491cb2ef744edc8cfde96a2801560d` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:15.142526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0491cb2ef744edc8cfde96a2801560d)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:15.143602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:15.143602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:31:15.143602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:31:15.144838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:15.154697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:15.469620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0491cb2ef744edc8cfde96a2801560d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:15.603373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0491cb2ef744edc8cfde96a2801560d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:15.603918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0491cb2ef744edc8cfde96a2801560d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:15.630540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0491cb2ef744edc8cfde96a2801560d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:15.645166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0491cb2ef744edc8cfde96a2801560d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:15.650684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 752/1507] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:31:14 2022] DELETE /identity/v3/users/f0491cb2ef744edc8cfde96a2801560d => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:31:15.655204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:15.655689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:15.655794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:15.715013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:31:15.716254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:15.793149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:15.806084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 756/1508] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:31:15.816927 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:15.818486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:15.920515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba709f-502c-4219-905d-b92a6b53ff7d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:15.976009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:15.994935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:16.001752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:16.001752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:16.001752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:16.001752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:16.106871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:16.108235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a277bbc-88cf-4660-8a7b-3d56cc85d294 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:16.128026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 753/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:31:16.586032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:16.587439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:31:16.600171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3d87d-f724-461b-aeed-912dfd9c54b7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:16.703242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:16.703916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:16.704141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:16.704443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] PATH_INFO: `/v3/projects/89d80f89d138415d9129b0a3d33c0977` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:16.714574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d80f89d138415d9129b0a3d33c0977)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:16.715599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:31:16.715599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:31:16.715599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:31:16.716844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:16.840731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d80f89d138415d9129b0a3d33c0977', 'request_id': 'req-67137337-8d76-4abd-8c8a-d3c02c37d8e8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:16.940175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d80f89d138415d9129b0a3d33c0977', 'request_id': 'req-67137337-8d76-4abd-8c8a-d3c02c37d8e8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:16.947275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d80f89d138415d9129b0a3d33c0977', 'request_id': 'req-67137337-8d76-4abd-8c8a-d3c02c37d8e8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:16.979267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] The token cache is being invalidate because project 89d80f89d138415d9129b0a3d33c0977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:31:16.979558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:16.980091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67137337-8d76-4abd-8c8a-d3c02c37d8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:31:16.981596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 757/1510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:31:16 2022] DELETE /identity/v3/projects/89d80f89d138415d9129b0a3d33c0977 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:31:26.124681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=62 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:26.977071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:32:48.027687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a62e-d72c-4f85-8342-071afb2dba9a None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:48.028290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a62e-d72c-4f85-8342-071afb2dba9a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:48.028290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a62e-d72c-4f85-8342-071afb2dba9a None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:48.029613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:32:48.034698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac3a50-7009-43c9-9341-bd5d7d9c5cdb None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:48.035130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac3a50-7009-43c9-9341-bd5d7d9c5cdb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:48.035227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac3a50-7009-43c9-9341-bd5d7d9c5cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:48.116169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d9ac3a50-7009-43c9-9341-bd5d7d9c5cdb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:32:48.117576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d9ac3a50-7009-43c9-9341-bd5d7d9c5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:32:48.222659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:32:48.376879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a62e-d72c-4f85-8342-071afb2dba9a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:32:48.379043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1258a62e-d72c-4f85-8342-071afb2dba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:32:48.501000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a62e-d72c-4f85-8342-071afb2dba9a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:32:48.506810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315be02-2365-43ec-9d7e-6ba090241295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:32:48.507425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315be02-2365-43ec-9d7e-6ba090241295 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:48.507701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315be02-2365-43ec-9d7e-6ba090241295 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:48.507971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315be02-2365-43ec-9d7e-6ba090241295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:48.516022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315be02-2365-43ec-9d7e-6ba090241295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:32:48.517684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315be02-2365-43ec-9d7e-6ba090241295 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:32:48.527655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 755/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:32:49.363294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e94708-4053-4ed0-81ba-a103c257ed6c None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:49.363868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e94708-4053-4ed0-81ba-a103c257ed6c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:49.363868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e94708-4053-4ed0-81ba-a103c257ed6c None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:49.365265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:32:49.371184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:49.371691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:49.371819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:49.393973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:32:49.395170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:32:49.404073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:32:49.552298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf77c4-3348-44ad-8114-6fbfd1d3fe58 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:49.552543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf77c4-3348-44ad-8114-6fbfd1d3fe58 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:49.552794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf77c4-3348-44ad-8114-6fbfd1d3fe58 None None] PATH_INFO: `/v3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:49.554623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 760/1516] 10.222.0.76 () {54 vars in 925 bytes} [Wed Aug 31 00:32:49 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:32:49.562345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:32:49.574894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c84a8-98ac-43db-8a72-6bc437fdb3bb None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:32:49.575343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d37490-c961-45c7-aed1-fb8840b0d18b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:49.575620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d37490-c961-45c7-aed1-fb8840b0d18b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:49.575894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d37490-c961-45c7-aed1-fb8840b0d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:49.580744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479393f4-f93e-4784-b7ef-89e21b270220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:32:49.581350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479393f4-f93e-4784-b7ef-89e21b270220 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:49.581630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479393f4-f93e-4784-b7ef-89e21b270220 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:49.581901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479393f4-f93e-4784-b7ef-89e21b270220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:49.590521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479393f4-f93e-4784-b7ef-89e21b270220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:32:49.592106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479393f4-f93e-4784-b7ef-89e21b270220 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:32:49.602474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 757/1517] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:32:49.637764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-95d37490-c961-45c7-aed1-fb8840b0d18b None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:32:49.638992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-95d37490-c961-45c7-aed1-fb8840b0d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:32:49.674575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:32:49.725728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 761/1518] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 31 00:32:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:32:58.992274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396a5e-e7ad-4bd0-aef0-212b9c087a29 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:58.992842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396a5e-e7ad-4bd0-aef0-212b9c087a29 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:58.992842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396a5e-e7ad-4bd0-aef0-212b9c087a29 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:58.994215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 758/1519] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 31 00:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:32:58.999117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd86ed1-bd98-4c05-9f27-98098b135b05 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:58.999500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd86ed1-bd98-4c05-9f27-98098b135b05 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:58.999768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd86ed1-bd98-4c05-9f27-98098b135b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:59.019945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-3dd86ed1-bd98-4c05-9f27-98098b135b05 None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:32:59.021206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3dd86ed1-bd98-4c05-9f27-98098b135b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:32:59.030427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 762/1520] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 31 00:32:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:32:59.038800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f396a5e-e7ad-4bd0-aef0-212b9c087a29 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:32:59.054525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f396a5e-e7ad-4bd0-aef0-212b9c087a29 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:32:59.059605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4f8a2-d9e8-420a-9dd9-a36abe368f27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:32:59.060206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4f8a2-d9e8-420a-9dd9-a36abe368f27 service placement] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:32:59.060521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4f8a2-d9e8-420a-9dd9-a36abe368f27 service placement] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:32:59.060792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4f8a2-d9e8-420a-9dd9-a36abe368f27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:32:59.070408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4f8a2-d9e8-420a-9dd9-a36abe368f27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:32:59.072058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4f8a2-d9e8-420a-9dd9-a36abe368f27 service placement] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:32:59.082378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 759/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:32:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:00.835062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f93c-d173-40b0-9458-27e32f6f18ca None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:00.835608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f93c-d173-40b0-9458-27e32f6f18ca None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:00.835608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f93c-d173-40b0-9458-27e32f6f18ca None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:00.837014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:33:00.841682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe17-feed-44cb-abe0-a5615033a8e9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:00.842100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe17-feed-44cb-abe0-a5615033a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:00.842202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe17-feed-44cb-abe0-a5615033a8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:00.861549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4017fe17-feed-44cb-abe0-a5615033a8e9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:33:00.862738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4017fe17-feed-44cb-abe0-a5615033a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:00.871058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:33:01.025452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f93c-d173-40b0-9458-27e32f6f18ca None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:01.040736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f93c-d173-40b0-9458-27e32f6f18ca None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:01.046304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866f18-2c98-476a-bbb9-30416be46f22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:01.046898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866f18-2c98-476a-bbb9-30416be46f22 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:01.047171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866f18-2c98-476a-bbb9-30416be46f22 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:01.047436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866f18-2c98-476a-bbb9-30416be46f22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:01.056301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866f18-2c98-476a-bbb9-30416be46f22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:01.057915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4866f18-2c98-476a-bbb9-30416be46f22 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:01.067638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 764/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:01.898299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8b8c5-2c18-4acd-88cc-2c147f8c00f4 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:01.898883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8b8c5-2c18-4acd-88cc-2c147f8c00f4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:01.898883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8b8c5-2c18-4acd-88cc-2c147f8c00f4 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:01.900148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:33:01.905308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e19fd-fb3b-429c-b3a4-69a6e0fc40b9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:01.905692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e19fd-fb3b-429c-b3a4-69a6e0fc40b9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:01.905850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e19fd-fb3b-429c-b3a4-69a6e0fc40b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:01.926168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-088e19fd-fb3b-429c-b3a4-69a6e0fc40b9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:33:01.927398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-088e19fd-fb3b-429c-b3a4-69a6e0fc40b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:01.935587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:33:02.093631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8b8c5-2c18-4acd-88cc-2c147f8c00f4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:02.112028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8b8c5-2c18-4acd-88cc-2c147f8c00f4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:02.118629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdb504-c99b-47eb-b8e9-b66ee3452368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:02.119238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdb504-c99b-47eb-b8e9-b66ee3452368 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:02.119513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdb504-c99b-47eb-b8e9-b66ee3452368 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:02.119779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdb504-c99b-47eb-b8e9-b66ee3452368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:02.130118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdb504-c99b-47eb-b8e9-b66ee3452368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:02.132903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdb504-c99b-47eb-b8e9-b66ee3452368 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:02.142880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 762/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:05.956031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3352e4-c703-4a67-b472-dfa701a95633 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:05.956612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3352e4-c703-4a67-b472-dfa701a95633 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:05.956612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3352e4-c703-4a67-b472-dfa701a95633 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:05.957968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:33:05.963536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0c7f2-dcd9-466f-a815-321fe3630f35 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:05.963990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0c7f2-dcd9-466f-a815-321fe3630f35 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:05.964094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0c7f2-dcd9-466f-a815-321fe3630f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:05.984258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2cc0c7f2-dcd9-466f-a815-321fe3630f35 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:33:05.985495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2cc0c7f2-dcd9-466f-a815-321fe3630f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:05.994856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:33:06.288231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3352e4-c703-4a67-b472-dfa701a95633 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:06.290344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fb3352e4-c703-4a67-b472-dfa701a95633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:06.371020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3352e4-c703-4a67-b472-dfa701a95633 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:06.374618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49113683-ef6e-4d18-ac5e-03e2e69935ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:06.375225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49113683-ef6e-4d18-ac5e-03e2e69935ab service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:06.375504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49113683-ef6e-4d18-ac5e-03e2e69935ab service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:06.375776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49113683-ef6e-4d18-ac5e-03e2e69935ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:06.385577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49113683-ef6e-4d18-ac5e-03e2e69935ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:06.387006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49113683-ef6e-4d18-ac5e-03e2e69935ab service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:06.396607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 767/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:08.342147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8205b34-45d4-4b1e-8c77-43ea5b030812 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:08.342754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8205b34-45d4-4b1e-8c77-43ea5b030812 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:08.342754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8205b34-45d4-4b1e-8c77-43ea5b030812 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:08.344237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:33:08.350444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e478958-e5cd-488c-9c23-25c726fed276 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:08.351411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e478958-e5cd-488c-9c23-25c726fed276 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:08.351411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e478958-e5cd-488c-9c23-25c726fed276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:08.374180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-8e478958-e5cd-488c-9c23-25c726fed276 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:33:08.375397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8e478958-e5cd-488c-9c23-25c726fed276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:08.384404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:33:08.675457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8205b34-45d4-4b1e-8c77-43ea5b030812 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:08.686142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8205b34-45d4-4b1e-8c77-43ea5b030812 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:08.689635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f83cb-aaf0-432e-8f25-df494c97d0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:08.690221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f83cb-aaf0-432e-8f25-df494c97d0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:08.690494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f83cb-aaf0-432e-8f25-df494c97d0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:08.690757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f83cb-aaf0-432e-8f25-df494c97d0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:08.700791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f83cb-aaf0-432e-8f25-df494c97d0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:08.702206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f83cb-aaf0-432e-8f25-df494c97d0f8 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:08.712002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 765/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:12.006766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46d08c-5be5-48de-a334-81514461b352 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:12.007374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46d08c-5be5-48de-a334-81514461b352 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:12.007374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46d08c-5be5-48de-a334-81514461b352 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:12.008701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:33:12.014623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb20ce-ab96-4bfa-83bd-0092fb36191e None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:12.015078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb20ce-ab96-4bfa-83bd-0092fb36191e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:12.015189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb20ce-ab96-4bfa-83bd-0092fb36191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:12.036593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2fbb20ce-ab96-4bfa-83bd-0092fb36191e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:33:12.037915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2fbb20ce-ab96-4bfa-83bd-0092fb36191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:12.046255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:33:12.610047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46d08c-5be5-48de-a334-81514461b352 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:12.611772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7b46d08c-5be5-48de-a334-81514461b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:12.707563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46d08c-5be5-48de-a334-81514461b352 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:12.712080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f3e8d-59f3-4ef9-8292-af7a1d23445b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:12.712729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3e8d-59f3-4ef9-8292-af7a1d23445b service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:12.713008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3e8d-59f3-4ef9-8292-af7a1d23445b service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:12.713276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f3e8d-59f3-4ef9-8292-af7a1d23445b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:12.720735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:33:12.723068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f3e8d-59f3-4ef9-8292-af7a1d23445b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:12.724650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f3e8d-59f3-4ef9-8292-af7a1d23445b service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:12.736852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 770/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:14.717693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544f41c-c90a-420c-bbad-565e987cca3d None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:14.718256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544f41c-c90a-420c-bbad-565e987cca3d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:14.718256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544f41c-c90a-420c-bbad-565e987cca3d None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:14.719555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:33:14.725826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dca84-bd10-482a-8a47-7ca31d5fde1f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:14.726351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dca84-bd10-482a-8a47-7ca31d5fde1f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:14.726351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dca84-bd10-482a-8a47-7ca31d5fde1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:14.747400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f76dca84-bd10-482a-8a47-7ca31d5fde1f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:33:14.748672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f76dca84-bd10-482a-8a47-7ca31d5fde1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:14.757267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:33:15.314594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544f41c-c90a-420c-bbad-565e987cca3d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:15.316373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4544f41c-c90a-420c-bbad-565e987cca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:33:15.330534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544f41c-c90a-420c-bbad-565e987cca3d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:15.334429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2aa7c-2bd7-4c7f-a92e-f2069d97a71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:15.335048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2aa7c-2bd7-4c7f-a92e-f2069d97a71d service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:15.335325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2aa7c-2bd7-4c7f-a92e-f2069d97a71d service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:15.335597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2aa7c-2bd7-4c7f-a92e-f2069d97a71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:15.346198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2aa7c-2bd7-4c7f-a92e-f2069d97a71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:15.347651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2aa7c-2bd7-4c7f-a92e-f2069d97a71d service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:15.358864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 768/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:33:24.741985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:33:25.355144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:42.800156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:42.801211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:42.801697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:42.833237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:42.834813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:42.850975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 772/1540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:34:42.862788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:42.863398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:42.863398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:42.894483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:42.895750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:42.914713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 769/1541] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:42.920076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:42.934625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a47ad-497c-4d29-a319-6b3de63a076c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:42.945060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:42.946073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:42.946574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:42.947187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:42.948110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:42.948657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:42.950554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:42.974843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 773/1542] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:42.983651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:42.997964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfdf19-d4ad-498b-a34b-3c82be2a96f5 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.005969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885ee1a-5be9-42ad-b179-6e5600b56099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.006589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885ee1a-5be9-42ad-b179-6e5600b56099 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.006871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885ee1a-5be9-42ad-b179-6e5600b56099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.007143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885ee1a-5be9-42ad-b179-6e5600b56099 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.007779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885ee1a-5be9-42ad-b179-6e5600b56099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.009763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885ee1a-5be9-42ad-b179-6e5600b56099 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.132233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 770/1543] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:42 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:43.138714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.157179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8b471-8525-465c-aaed-455ac61d4590 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.251678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819a541-509c-4e0e-a5f9-47c191348ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.252786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a541-509c-4e0e-a5f9-47c191348ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.253280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a541-509c-4e0e-a5f9-47c191348ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.253772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a541-509c-4e0e-a5f9-47c191348ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.255157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a541-509c-4e0e-a5f9-47c191348ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.257678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a541-509c-4e0e-a5f9-47c191348ae4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.384831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.385688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.386189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.405640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 774/1544] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:43.415024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d036f1-43b6-456b-8d82-6637402e63b1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.415024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d036f1-43b6-456b-8d82-6637402e63b1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.415024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d036f1-43b6-456b-8d82-6637402e63b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.436197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.444429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.455935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-22d036f1-43b6-456b-8d82-6637402e63b1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.459553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-22d036f1-43b6-456b-8d82-6637402e63b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.464534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 771/1545] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.465685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.479849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 775/1546] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.483640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66afd1-535c-44c9-8624-85912e72bf76 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.485360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66afd1-535c-44c9-8624-85912e72bf76 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.489125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f23531-b7e7-457b-8051-c1dcb53d2870 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.489724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66afd1-535c-44c9-8624-85912e72bf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.501815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30db12-ab14-4447-871a-d51502c17554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.506810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30db12-ab14-4447-871a-d51502c17554 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.506810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30db12-ab14-4447-871a-d51502c17554 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.506810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30db12-ab14-4447-871a-d51502c17554 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.506810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30db12-ab14-4447-871a-d51502c17554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.509798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30db12-ab14-4447-871a-d51502c17554 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.531087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1a66afd1-535c-44c9-8624-85912e72bf76 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.534926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1a66afd1-535c-44c9-8624-85912e72bf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.539561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 772/1547] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:43.543307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a51027-bda9-431d-a86e-af88cc2a6293 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.543873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a51027-bda9-431d-a86e-af88cc2a6293 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.544525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a51027-bda9-431d-a86e-af88cc2a6293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.554590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 776/1548] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.555987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adeed1-2a2c-4a58-8370-bf26c82ea867 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.556299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adeed1-2a2c-4a58-8370-bf26c82ea867 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.557132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adeed1-2a2c-4a58-8370-bf26c82ea867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.568844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-24a51027-bda9-431d-a86e-af88cc2a6293 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.570044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-24a51027-bda9-431d-a86e-af88cc2a6293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.578019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c3adeed1-2a2c-4a58-8370-bf26c82ea867 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.578971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 773/1549] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.580198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.580523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.580791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.584780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c3adeed1-2a2c-4a58-8370-bf26c82ea867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.598159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 777/1550] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.599540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be5664-5ca8-444b-90b2-f0827b2095e8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.599834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be5664-5ca8-444b-90b2-f0827b2095e8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.600103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be5664-5ca8-444b-90b2-f0827b2095e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.636811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.637099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d3be5664-5ca8-444b-90b2-f0827b2095e8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.637949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.638263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d3be5664-5ca8-444b-90b2-f0827b2095e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.645888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 774/1551] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.646652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.648467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 778/1552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.649927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70034-90c8-4a6e-b12b-d51c8dc419ba None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.651244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70034-90c8-4a6e-b12b-d51c8dc419ba None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.652106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70034-90c8-4a6e-b12b-d51c8dc419ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.657453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2e138-52c4-44b1-b1a7-5ef0541c6bba None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.662065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.662675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.662949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.663242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.663910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.665539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.674266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2cf70034-90c8-4a6e-b12b-d51c8dc419ba None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.675473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2cf70034-90c8-4a6e-b12b-d51c8dc419ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.676351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 775/1553] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:43.677107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.683867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 779/1554] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.685109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a65c0-c896-4f2f-bb13-30a6a46bfc07 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.685385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a65c0-c896-4f2f-bb13-30a6a46bfc07 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.685650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a65c0-c896-4f2f-bb13-30a6a46bfc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.694466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b85fd-4be8-49fb-b5aa-9339b8decebc admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.699310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.699913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.700190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.700495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.701140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:43.701440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.702466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:34:43.702466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:34:43.702466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:34:43.703349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.704761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-4b9a65c0-c896-4f2f-bb13-30a6a46bfc07 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.705948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4b9a65c0-c896-4f2f-bb13-30a6a46bfc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.714547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 780/1555] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.715738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db905018-fd89-4713-b85c-24aa9c481ecf None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.716214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 776/1556] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:43.716914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.722978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db905018-fd89-4713-b85c-24aa9c481ecf None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.723314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db905018-fd89-4713-b85c-24aa9c481ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.726787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53b94-f354-49bf-8fb3-f936048e005b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.733493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.734104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.734387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.734660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.735311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:43.735615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.737298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ec206-d82d-4ef6-8609-f5485d57139f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.743354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-db905018-fd89-4713-b85c-24aa9c481ecf None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.744646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-db905018-fd89-4713-b85c-24aa9c481ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.750345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 777/1557] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:43.751730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.752043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.752340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.753535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 781/1558] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.757910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af189-fbea-4b01-b7bb-aa9021ca751a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.758192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af189-fbea-4b01-b7bb-aa9021ca751a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.758560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af189-fbea-4b01-b7bb-aa9021ca751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.772449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.773665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.779363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fc3af189-fbea-4b01-b7bb-aa9021ca751a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.780578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fc3af189-fbea-4b01-b7bb-aa9021ca751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.783747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 778/1559] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.785136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.789200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 782/1560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.790803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.791608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.791824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.796455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78e18d-682e-40c1-90f8-398b7914ada7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.801063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.801898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.802179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.802448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] PATH_INFO: `/v3/projects/c0528602037241c98f36d12984aa26a9/users/703b8e63fa4449dfb85de594a8d2e5fe/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.812313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:43.813561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:43.817118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0528602037241c98f36d12984aa26a9, user_id=703b8e63fa4449dfb85de594a8d2e5fe, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.819787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.822516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 783/1561] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:43.824165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.833856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d2e6f-3744-4a5d-830f-ab74be58e67c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.839480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.841869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.841869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.841869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.841869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:43.841869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.843542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.855817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 784/1562] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:43.856744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.866334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd41e3-ac29-4c66-b7a0-a1feee8aed1a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.871147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:34:43.874886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:34:43.875482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196857d8-95b4-4f75-9074-9d49faa6e368 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.903619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 779/1563] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:43 2022] PUT /identity/v3/projects/c0528602037241c98f36d12984aa26a9/users/703b8e63fa4449dfb85de594a8d2e5fe/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:43.903619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:43.966983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada17753-02d0-409e-a882-63b46bfae398 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:43.971800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:43.972440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.972713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.972985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:43.973615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:43.975532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:43.988473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 785/1564] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:43.991676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:43.991676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:43.991676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.011119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:44.012308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:44.043175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:44.071386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 786/1565] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:44.078957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.079098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 780/1566] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:43 2022] POST /identity/v3/projects => generated 393 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.079813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.082134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf2fc-64c6-41af-872d-b2c6d87b961c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.091759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:44.141642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.142264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.142546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.142838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.143515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:44.143818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.144062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0392a9-a630-499a-9c08-fc9b921d28af admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.145497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.148968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.149635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.149920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.150823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.152117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:44.153075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.157464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.159834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 787/1567] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:44.161148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.171640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314a4d5-42b0-4acb-bc68-b2aa7fb286c9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.182105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 781/1568] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:44.182374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.182552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.182723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.182723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.182723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.182723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:44.182723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.182723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d191be-4991-4af1-83e4-3c2e58c7a6dd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.192070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e2785-3e62-4681-80cb-1ca2993578d7 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.203297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 788/1569] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:44.203297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.203297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.203297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.203773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.203773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.203773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.203773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.204140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.204140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.216712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:44.217909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:44.225842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 789/1570] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:44.228495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.238285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f906cf-daaa-4df9-98db-22bd953f6e72 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.242870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.245378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.245378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.245378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.245692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.247341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.263589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 782/1571] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.264461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.317114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 790/1572] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.317114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.324946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538acf1-b7d5-44de-bbbe-95d4abd4780d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.329714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.330332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.330521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5318-a434-4869-b879-1a78475ae8c0 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.330707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.330882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.331511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.333438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.335444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.336054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.336361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.336638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.337281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:44.337674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.339583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.351665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 783/1573] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:44.352627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.362486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595451e6-a880-4ce5-a259-77a7bae04e0f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.367084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.367666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.367947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.368221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.370173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.371738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.434477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 791/1574] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.435211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.445109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267b376-1439-44ad-bb17-6e7c9835a50e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.450156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.451018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.451301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.451571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.452198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.454215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.457137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 784/1575] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.458135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.512606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:44.522822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001db0-1cca-4aee-9f0b-45efea258b67 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.526488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 792/1576] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/projects => generated 405 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.527303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.531760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.532404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.532687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.532960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.533598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.535518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.584646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3542c7-4adb-4dc5-8dcd-5d08e9e6041f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.589066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.589677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.589955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.590231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.595402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:44.621882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 785/1577] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.622544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.654602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.656048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.666296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 793/1578] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:44.667524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.678187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0a2b6-3597-461d-88de-c76ba9fafa00 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.688883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b014d-16f7-49f1-a857-b0dde0a69bb6 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.693669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.694283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.694561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.694846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.695453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.696578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:34:44.696578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:34:44.696578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:34:44.699111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.738880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.739518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.739791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.740058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.740715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.742290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.753552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 794/1579] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:44.756545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.766170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47b846-9cfb-4e0a-9e8f-d32487a16fc2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.770854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.771442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.771714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.771980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.772626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.774626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.785387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 786/1580] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.785387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.797802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30819dfc-72c7-4b60-927b-e7a24a3a0258 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.802868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.803502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.803781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.804052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.804741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.806483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.816786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 787/1581] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:44.817588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.828339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235adae-cff6-4b1c-8eba-0df33ee2e86f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.833168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.833791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.834072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.834347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.834969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.836976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.874557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 795/1582] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.882141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.939239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b450d2-4af4-45de-b8ea-cada1c915278 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:44.943976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:44.944612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:44.944887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:44.945154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:44.945785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:44.948288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:44.967363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 788/1583] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:44.970517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:44.986786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad32c8-8c03-4b2f-9a91-77755a6a9def admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.006723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.007428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.007781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.008125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.008893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.011003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.052102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 796/1584] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/users => generated 303 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:45.052102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.060391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9727fc6-ecd8-4c5a-ad70-7549fd080167 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.065184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.065826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.066104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.066374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.067038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.068661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.070000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:45.079821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 797/1585] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.080805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.090465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb850-756d-467c-bb47-c1ff3fd853ce admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.095256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.097902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.097902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.097902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.097902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.098500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.108888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 798/1586] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.109817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.120002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf1130-5055-48fc-a1c7-894e16686889 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.126563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 789/1587] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:44 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:45.126563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.128285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.128907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.129195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.129465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.130097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.131683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.135606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40f35-7499-4955-acc2-7e53b309bd69 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.140615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.141211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.149635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.149635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.149635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.150066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 799/1588] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.150066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.150370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.155578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180dde-49c1-40ca-9e74-56993c7a73a7 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.161094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.161690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.161970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.162240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.162874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.164513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.175821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 800/1589] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.178403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.187319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a7e27-d837-40e8-b7d9-d18caad972af admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.192851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.193395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.193676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.193950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.194617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.196234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.207775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 801/1590] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.208689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.218459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54185043-50fc-492e-809c-0fe791b779ef admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.223409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.224064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.224365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.224639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] PATH_INFO: `/v3/projects/e931d084c75b4dd3871922e92e278f88/users/d3174cfa2b474a7b90d337a16fe6e522/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.234601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 790/1591] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:45 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:45.235787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.242827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e931d084c75b4dd3871922e92e278f88, user_id=d3174cfa2b474a7b90d337a16fe6e522, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.245058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:34:45.245058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:34:45.245058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:34:45.246732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.247459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901cc7-a9d6-4aad-ace0-e16dd1fe0b4e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.252755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.253566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.253692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.253891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.254534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.256158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.268281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 791/1592] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.269144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.281109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58c53c-30e8-46f1-94ef-41ddb0976865 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.286218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.286824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.287107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.287378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.288029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.293314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.301384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 792/1593] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.302336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.313528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18ae6-086e-4c28-a2b6-b0ffaffd437a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.319401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 802/1594] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:45 2022] PUT /identity/v3/projects/e931d084c75b4dd3871922e92e278f88/users/d3174cfa2b474a7b90d337a16fe6e522/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:45.336800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.369358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:45.387020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.388000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.388910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.388910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] PATH_INFO: `/v3/projects/d8aa6e0e8e6b492f8e74eea0e830c0bc/users/77d23c99308c414f939761b07d329e03/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.404822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8aa6e0e8e6b492f8e74eea0e830c0bc, user_id=77d23c99308c414f939761b07d329e03, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.406733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624883e5-78c7-4bbc-85c2-20ca1bb471be admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.407767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.412017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.412017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.412317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.412637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.417441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.417441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.429167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 803/1595] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.432000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.441149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87411500-ce75-4d6d-8e26-ab0cadb21815 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.445831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.446368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.446642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.446946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.447890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.449524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.460181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 804/1596] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.461282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.471572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b9828-899b-42e7-bbfb-8bbde46668c2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.476312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.476946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.477240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.477557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.482321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.482321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.497876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 805/1597] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.497876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.505136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8debc-f718-4f7b-b5f6-d38a568d0f32 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.510564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.511241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.517688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.517688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] PATH_INFO: `/v3/projects/632c182513b04837be04a6f87d20ae9d/users/7097afa9715849e0b18e46dc8f8b3c79/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.521860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 793/1598] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:45 2022] PUT /identity/v3/projects/d8aa6e0e8e6b492f8e74eea0e830c0bc/users/77d23c99308c414f939761b07d329e03/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:45.521860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.530702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=632c182513b04837be04a6f87d20ae9d, user_id=7097afa9715849e0b18e46dc8f8b3c79, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.533812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.590720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd15d3-977d-4410-b4db-ed79a5d44666 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.595356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3b7e6-0fc6-41d7-8c31-176fb1304919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.595954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b7e6-0fc6-41d7-8c31-176fb1304919 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.596230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b7e6-0fc6-41d7-8c31-176fb1304919 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.596523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b7e6-0fc6-41d7-8c31-176fb1304919 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.597158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b7e6-0fc6-41d7-8c31-176fb1304919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.598757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b7e6-0fc6-41d7-8c31-176fb1304919 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.609270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 794/1599] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.610495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.610778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.611048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.617952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 806/1600] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:45 2022] PUT /identity/v3/projects/632c182513b04837be04a6f87d20ae9d/users/7097afa9715849e0b18e46dc8f8b3c79/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:45.618940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.640778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] MFA Rules not processed for user `d3174cfa2b474a7b90d337a16fe6e522`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:45.642072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:45.675202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:45.699881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee69cf7-8e2c-415c-876e-12efba08a7ec admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.705131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.705938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.706218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.706494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] PATH_INFO: `/v3/projects/606c24bee18a4454ab672cfb6a395286/users/2a927401086b4a649919a6071b4a0fd9/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.729791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 795/1601] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:45.729791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.731445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=606c24bee18a4454ab672cfb6a395286, user_id=2a927401086b4a649919a6071b4a0fd9, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.734179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.751837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec97fe-549b-4efe-9a52-2e471b5285dd None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.756461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0f98f-4c63-45d8-884b-462b2f927369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.757190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0f98f-4c63-45d8-884b-462b2f927369 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.757535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0f98f-4c63-45d8-884b-462b2f927369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.757867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0f98f-4c63-45d8-884b-462b2f927369 admin admin] PATH_INFO: `/v3/projects/751d12acbdb540539718f10f4edf9213/users/8340e8a542734430b981288ee55e8f68/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.773746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0f98f-4c63-45d8-884b-462b2f927369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751d12acbdb540539718f10f4edf9213, user_id=8340e8a542734430b981288ee55e8f68, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.777707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0f98f-4c63-45d8-884b-462b2f927369 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.793711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 807/1602] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:45 2022] PUT /identity/v3/projects/606c24bee18a4454ab672cfb6a395286/users/2a927401086b4a649919a6071b4a0fd9/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:45.794552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.827999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 796/1603] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:45 2022] PUT /identity/v3/projects/751d12acbdb540539718f10f4edf9213/users/8340e8a542734430b981288ee55e8f68/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:45.829315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.829587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.829862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.850649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:45.851906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:45.861859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede771b-7d1c-416d-b50f-71563908f3fb admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.875049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:45.927054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47c281-2ec1-4cad-b24c-767f03963e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.927659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47c281-2ec1-4cad-b24c-767f03963e8f admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.927929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47c281-2ec1-4cad-b24c-767f03963e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.928201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47c281-2ec1-4cad-b24c-767f03963e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.928915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47c281-2ec1-4cad-b24c-767f03963e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.930531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47c281-2ec1-4cad-b24c-767f03963e8f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.937774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 797/1604] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:45.938729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:45.943653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 808/1605] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:45.945331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9349a-b9c1-418b-8f98-a0cb2556f452 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.946931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9349a-b9c1-418b-8f98-a0cb2556f452 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.947223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9349a-b9c1-418b-8f98-a0cb2556f452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.950523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68a9f5-9893-4f35-ac0a-70aed68c9c3d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:45.956063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbee5dc-f01d-4df9-a15c-6b56439a10d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:45.957056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee5dc-f01d-4df9-a15c-6b56439a10d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:45.957357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee5dc-f01d-4df9-a15c-6b56439a10d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:45.957701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee5dc-f01d-4df9-a15c-6b56439a10d5 admin admin] PATH_INFO: `/v3/projects/b05557d292f240d690687db45cfcbffc/users/7472ccd1202a4bf0b91e75fece3ca913/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:45.977484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbee5dc-f01d-4df9-a15c-6b56439a10d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05557d292f240d690687db45cfcbffc, user_id=7472ccd1202a4bf0b91e75fece3ca913, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:45.980261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9ac9349a-b9c1-418b-8f98-a0cb2556f452 None None] MFA Rules not processed for user `7097afa9715849e0b18e46dc8f8b3c79`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:45.981313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbee5dc-f01d-4df9-a15c-6b56439a10d5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:45.982046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9ac9349a-b9c1-418b-8f98-a0cb2556f452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.046862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:46.058235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 809/1606] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:46.059449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d8bd-9832-45d3-bdfb-2b263e8f1ffa None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.059809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d8bd-9832-45d3-bdfb-2b263e8f1ffa None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.060225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d8bd-9832-45d3-bdfb-2b263e8f1ffa None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.061758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 810/1607] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 00:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:34:46.066069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.066069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.066069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.076852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 798/1608] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:45 2022] PUT /identity/v3/projects/b05557d292f240d690687db45cfcbffc/users/7472ccd1202a4bf0b91e75fece3ca913/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:46.078548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.081270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.081635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.089544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:46.094131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.120952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] MFA Rules not processed for user `8340e8a542734430b981288ee55e8f68`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:46.122895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.169973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 811/1609] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:46.171117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.182282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffdbee-2141-4f4f-9fdd-0ca88f77f1d9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.187545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.188292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.188667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.188936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] PATH_INFO: `/v3/projects/ec5893d9a55f4ed28f6579397d0d6f21/users/45422a67a78f4784abc0f18ad84778f3/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.204885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5893d9a55f4ed28f6579397d0d6f21, user_id=45422a67a78f4784abc0f18ad84778f3, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.207461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.210284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 799/1610] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:46.211289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.214950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.281994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1a6c6-5219-49d7-a4e8-4cc1a5940df5 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.287470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce2d81-fb33-427b-a46a-b712a0058b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.288265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce2d81-fb33-427b-a46a-b712a0058b31 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.288578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce2d81-fb33-427b-a46a-b712a0058b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.288853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce2d81-fb33-427b-a46a-b712a0058b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.299555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce2d81-fb33-427b-a46a-b712a0058b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.300948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce2d81-fb33-427b-a46a-b712a0058b31 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.313252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 800/1611] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:46.314592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc95f3-d475-4837-8c99-ae8981e50d96 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.317756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc95f3-d475-4837-8c99-ae8981e50d96 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.317756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc95f3-d475-4837-8c99-ae8981e50d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.323227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 812/1612] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:46 2022] PUT /identity/v3/projects/ec5893d9a55f4ed28f6579397d0d6f21/users/45422a67a78f4784abc0f18ad84778f3/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:46.325453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.392981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-43bc95f3-d475-4837-8c99-ae8981e50d96 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:46.394934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-43bc95f3-d475-4837-8c99-ae8981e50d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.409104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6a07b-aba3-4dd2-b94f-179801016b8a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.414629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.415254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.418601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.419040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.419857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.421847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.422774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:46.496040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 801/1613] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:46.497610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.497764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.498034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.529470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] MFA Rules not processed for user `7472ccd1202a4bf0b91e75fece3ca913`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:46.530683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.549666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 813/1614] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:46.550542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.616681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 802/1615] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:46.620190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.629631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367fde-4b5f-4c01-a871-2d18bde87a20 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.634528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2a76-eb2b-4388-b07d-b5ffb994488c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.636718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2a76-eb2b-4388-b07d-b5ffb994488c service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.637490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2a76-eb2b-4388-b07d-b5ffb994488c service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.637992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2a76-eb2b-4388-b07d-b5ffb994488c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.654196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:46.659932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fa164-eb88-46f5-8f0e-736d09541b52 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.747960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2a76-eb2b-4388-b07d-b5ffb994488c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.749954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2a76-eb2b-4388-b07d-b5ffb994488c service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.750663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.751237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.751518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.752212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.752755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.754879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.766137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 814/1616] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:46.767661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.768151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.774615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.792823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:46.794330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.806208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 815/1617] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:46.806208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.859543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 803/1618] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/projects => generated 403 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:46.861446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.902190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04d3a7-ab66-4aca-8346-07408a373d99 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.905301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:46.949756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e0ce5-45b0-4c3b-bd25-bd2978ce79c3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.957377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f054eb5-a75f-4802-b4c9-2ac807c5315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.958453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f054eb5-a75f-4802-b4c9-2ac807c5315f admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.958839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f054eb5-a75f-4802-b4c9-2ac807c5315f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.959358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f054eb5-a75f-4802-b4c9-2ac807c5315f admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.960916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f054eb5-a75f-4802-b4c9-2ac807c5315f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.964664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f054eb5-a75f-4802-b4c9-2ac807c5315f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.986335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.987623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.987901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.988195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.060152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.062258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.072123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 804/1619] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:46 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:47.072123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e436-9373-4428-95e8-c7a7726836ca None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.072123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e436-9373-4428-95e8-c7a7726836ca None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.072123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413e436-9373-4428-95e8-c7a7726836ca None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.073980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 805/1620] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 00:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:34:47.074807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e436-9373-4428-95e8-c7a7726836ca None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.083024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 816/1621] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:47.083995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.095567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae0a5f-4188-4896-ac6f-0cad62b1bd5f service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.105079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.106005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.106334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.106605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.107944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.112227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.156525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413e436-9373-4428-95e8-c7a7726836ca None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.160932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de48b-9a6b-48c2-bf25-db19d71b72da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.161924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de48b-9a6b-48c2-bf25-db19d71b72da service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.162267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de48b-9a6b-48c2-bf25-db19d71b72da service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.162737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de48b-9a6b-48c2-bf25-db19d71b72da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.173730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de48b-9a6b-48c2-bf25-db19d71b72da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.183466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 817/1622] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:47 2022] POST /identity/v3/users => generated 308 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:47.184189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.185011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de48b-9a6b-48c2-bf25-db19d71b72da service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.203692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fd44-4cc3-4e27-954d-71ab712fd786 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.206158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 806/1623] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:47.207596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.207732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.208531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.217355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.217997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.218274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.218547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.219197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.220932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.235298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 818/1624] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:47.236236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.250319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a052f-b96b-4648-9fb9-1788a264ee5a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.256297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.259868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.259868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.259868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.259868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.259868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.273156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 819/1625] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:47.274295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.277545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:47.278854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:47.290487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc716-0113-4375-ae67-81e071529862 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.424225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 807/1626] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:47.424225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.440531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.440531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.440531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.440531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] PATH_INFO: `/v3/projects/eef5188ba89c44b0bbfa6b431d34dd28/users/6091338ea5014c58996a472ed0c7a1e4/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.441878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71385-4c02-4cf7-9305-77a3c7b09a72 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.441878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ea2fb-2af8-4d7d-9958-c618e78d4949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.441878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ea2fb-2af8-4d7d-9958-c618e78d4949 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.441878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ea2fb-2af8-4d7d-9958-c618e78d4949 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.442628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ea2fb-2af8-4d7d-9958-c618e78d4949 admin admin] PATH_INFO: `/v3/projects/6da1377b83bc48c28706ac7d0b9e6360/users/6e157fdc3bcd4ee88271ccc372d42660/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.442921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:47.442921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef5188ba89c44b0bbfa6b431d34dd28, user_id=6091338ea5014c58996a472ed0c7a1e4, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.442921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.443461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ea2fb-2af8-4d7d-9958-c618e78d4949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da1377b83bc48c28706ac7d0b9e6360, user_id=6e157fdc3bcd4ee88271ccc372d42660, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.443461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ea2fb-2af8-4d7d-9958-c618e78d4949 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.512605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 820/1627] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:47 2022] PUT /identity/v3/projects/eef5188ba89c44b0bbfa6b431d34dd28/users/6091338ea5014c58996a472ed0c7a1e4/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:47.512605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.536792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 808/1628] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:47 2022] PUT /identity/v3/projects/6da1377b83bc48c28706ac7d0b9e6360/users/6e157fdc3bcd4ee88271ccc372d42660/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:47.541489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8795-3ec1-447d-a1fd-9f27e6fe7800 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.541489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8795-3ec1-447d-a1fd-9f27e6fe7800 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.541489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8795-3ec1-447d-a1fd-9f27e6fe7800 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.544910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 809/1629] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 00:34:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:34:47.545967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e8795-3ec1-447d-a1fd-9f27e6fe7800 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.655299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e8795-3ec1-447d-a1fd-9f27e6fe7800 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.673294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.673294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.675040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.675609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.676378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.679350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.699723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 810/1630] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:47.707575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.724724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc7254-12c7-4823-861f-02d2203848d0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.727250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8abf9-b03b-4817-8fb5-c4e014fdb310 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.746015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd096244-fc8c-4f8d-afe3-ead1638e5750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.746633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd096244-fc8c-4f8d-afe3-ead1638e5750 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.746953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd096244-fc8c-4f8d-afe3-ead1638e5750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.747178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd096244-fc8c-4f8d-afe3-ead1638e5750 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.747814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd096244-fc8c-4f8d-afe3-ead1638e5750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.749483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd096244-fc8c-4f8d-afe3-ead1638e5750 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.769247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 811/1631] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:47.770975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.770975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.771262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.812219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:47.815050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:47.856825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.858480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.858480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.858480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.866046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:47.928894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 812/1632] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:47.932105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.953329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833155b9-90cb-419c-87a8-0fca95ac91b5 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:47.959665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:47.961023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:47.961307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:47.965092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.967025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.972118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] PATH_INFO: `/v3/projects/eef5188ba89c44b0bbfa6b431d34dd28/users/6091338ea5014c58996a472ed0c7a1e4/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:47.979761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef5188ba89c44b0bbfa6b431d34dd28, user_id=6091338ea5014c58996a472ed0c7a1e4, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:47.980107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 821/1633] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:47.981279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:47.982520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:47.998640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33a3f1-5719-48a5-9cc6-06c0b18deb92 service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:48.029737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 813/1634] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:47 2022] PUT /identity/v3/projects/eef5188ba89c44b0bbfa6b431d34dd28/users/6091338ea5014c58996a472ed0c7a1e4/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:48.031521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:48.112020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:48.165980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f5f99-e516-452c-b8a2-e7a25a9c41a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:48.167910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f5f99-e516-452c-b8a2-e7a25a9c41a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.168175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f5f99-e516-452c-b8a2-e7a25a9c41a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.168523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f5f99-e516-452c-b8a2-e7a25a9c41a4 admin admin] PATH_INFO: `/v3/projects/6da1377b83bc48c28706ac7d0b9e6360/users/6e157fdc3bcd4ee88271ccc372d42660/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.174472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f5f99-e516-452c-b8a2-e7a25a9c41a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da1377b83bc48c28706ac7d0b9e6360, user_id=6e157fdc3bcd4ee88271ccc372d42660, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:48.179054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132c20d-39aa-4139-82e1-6e742d10b3a6 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:48.182212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f5f99-e516-452c-b8a2-e7a25a9c41a4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:48.191570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049dc6e-0b41-4279-8234-ab7933d120c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:48.192199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049dc6e-0b41-4279-8234-ab7933d120c6 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.192584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049dc6e-0b41-4279-8234-ab7933d120c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.193177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049dc6e-0b41-4279-8234-ab7933d120c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.216935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 822/1635] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:47 2022] PUT /identity/v3/projects/6da1377b83bc48c28706ac7d0b9e6360/users/6e157fdc3bcd4ee88271ccc372d42660/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:48.218305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d503da-648c-49a4-836e-ed92d4d29481 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.218591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d503da-648c-49a4-836e-ed92d4d29481 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.218858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d503da-648c-49a4-836e-ed92d4d29481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.287279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e9d503da-648c-49a4-836e-ed92d4d29481 None None] MFA Rules not processed for user `6091338ea5014c58996a472ed0c7a1e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:48.290032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e9d503da-648c-49a4-836e-ed92d4d29481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:48.329837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049dc6e-0b41-4279-8234-ab7933d120c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:48.331678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049dc6e-0b41-4279-8234-ab7933d120c6 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:48.354637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 814/1636] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:48.356339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d715b3f-ba0c-47fd-80b8-92cec6932f91 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.356923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d715b3f-ba0c-47fd-80b8-92cec6932f91 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.639842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d715b3f-ba0c-47fd-80b8-92cec6932f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.639842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3d715b3f-ba0c-47fd-80b8-92cec6932f91 None None] MFA Rules not processed for user `6e157fdc3bcd4ee88271ccc372d42660`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:48.639842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3d715b3f-ba0c-47fd-80b8-92cec6932f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:48.652441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 823/1637] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:48.652441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a98a8-dda7-4071-8058-a9ae382460eb None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.652441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a98a8-dda7-4071-8058-a9ae382460eb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.652441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a98a8-dda7-4071-8058-a9ae382460eb None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.652441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 824/1638] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 00:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:34:48.652441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a98a8-dda7-4071-8058-a9ae382460eb None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:48.663056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 815/1639] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:48.663056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.663056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.663056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.674382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a98a8-dda7-4071-8058-a9ae382460eb None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:48.684638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:48.684638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:48.691125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:48.691125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.691125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.691125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.693863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:48.693863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:48.693863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 825/1640] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:48.811187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 816/1641] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:48.875258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:48.888462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c44b87-9971-40ee-a2b6-7dccd49f2350 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:48.897100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:48.897729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:48.898003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:48.898270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:48.923362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:48.925942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:48.985218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 826/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:48 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:49.082733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:49.128790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b3504-ed10-43b3-97ff-c162ab6e4aaa None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:49.142431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:49.144336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:49.147812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:49.150505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:49.151253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:49.182898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:49.189013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:49.254642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 817/1643] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:49.563138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:49.571608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6bc39-ad15-4699-a264-6baa4f447c48 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:49.576278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:49.577072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:49.577442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:49.577935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:49.657367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:49.658994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:49.663020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:49.671039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 827/1644] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:49.977433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:50.133493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea73078-d377-4932-93ce-bd4ef8f9d808 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:50.151739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:50.156030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:50.156803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:50.157083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:50.160790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] PATH_INFO: `/v3/users/7472ccd1202a4bf0b91e75fece3ca913` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:50.161402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7472ccd1202a4bf0b91e75fece3ca913)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:50.166558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:50.183363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a45b-9713-4f84-bfd5-c04981e5b1d7 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:50.196949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:50.199230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:50.200440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:50.200860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] PATH_INFO: `/v3/users/8340e8a542734430b981288ee55e8f68` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:50.205538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8340e8a542734430b981288ee55e8f68)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:50.208862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: WARNING py.warnings [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:34:50.208862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: warnings.warn(msg) Aug 31 00:34:50.208862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: Aug 31 00:34:50.218973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:50.427544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7472ccd1202a4bf0b91e75fece3ca913'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.512609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7472ccd1202a4bf0b91e75fece3ca913'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.513356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7472ccd1202a4bf0b91e75fece3ca913'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.533392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7472ccd1202a4bf0b91e75fece3ca913'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.546824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8340e8a542734430b981288ee55e8f68'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.551405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a90e5fc-7d09-4e22-a392-60570ef51627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7472ccd1202a4bf0b91e75fece3ca913'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.566723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 818/1645] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:34:49 2022] DELETE /identity/v3/users/7472ccd1202a4bf0b91e75fece3ca913 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:50.577847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:50.578695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:50.579196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:50.597368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8340e8a542734430b981288ee55e8f68'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.616088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:50.617372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:50.619783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8340e8a542734430b981288ee55e8f68'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.707245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8340e8a542734430b981288ee55e8f68'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.707929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-742aff3f-8a32-4aae-abde-e3b7e36e2882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8340e8a542734430b981288ee55e8f68'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:50.720682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 828/1646] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:34:50 2022] DELETE /identity/v3/users/8340e8a542734430b981288ee55e8f68 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:50.723722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:50.724009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:50.724285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:50.743417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 819/1647] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:34:50.755279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:50.756970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:50.786836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:50.787648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:50.831082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:50.894462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd3036-a7ee-473b-92e5-9099f5af20ed None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:50.936363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 829/1648] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:34:50.963645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:50.964612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:51.038659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:51.039279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:51.039558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:51.039831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:51.068365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:51.068365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:51.093294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:51.110157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 820/1649] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:34:51.163523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c7e91-4839-4db8-a7a0-7921c0389168 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:51.173872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5170248-baae-4c72-99ef-e20dac39c0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:51.174509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5170248-baae-4c72-99ef-e20dac39c0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:51.174784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5170248-baae-4c72-99ef-e20dac39c0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:51.175055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5170248-baae-4c72-99ef-e20dac39c0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:51.208259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5170248-baae-4c72-99ef-e20dac39c0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:51.209718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5170248-baae-4c72-99ef-e20dac39c0d1 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:51.238840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 830/1650] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:52.453386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:52.455452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:52.485724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f35d9-e307-4769-bd41-ed099daab237 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:52.491393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:52.494340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:52.494340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:52.494340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] PATH_INFO: `/v3/projects/b05557d292f240d690687db45cfcbffc` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:52.498689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05557d292f240d690687db45cfcbffc)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:52.499903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: WARNING py.warnings [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:34:52.499903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: warnings.warn(msg) Aug 31 00:34:52.499903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: Aug 31 00:34:52.501184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:52.535804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:52.763383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05557d292f240d690687db45cfcbffc', 'request_id': 'req-84be91ce-5300-43ba-9b9c-dd7e30e8314f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:52.772438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05557d292f240d690687db45cfcbffc', 'request_id': 'req-84be91ce-5300-43ba-9b9c-dd7e30e8314f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:52.795785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05557d292f240d690687db45cfcbffc', 'request_id': 'req-84be91ce-5300-43ba-9b9c-dd7e30e8314f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:52.960950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] The token cache is being invalidate because project b05557d292f240d690687db45cfcbffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:34:52.963174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:52.966427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:52.970766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 821/1651] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:34:52 2022] DELETE /identity/v3/projects/b05557d292f240d690687db45cfcbffc => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:52.984076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac3b3-3307-4fe6-8ada-aad0c898e727 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:52.984076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac3b3-3307-4fe6-8ada-aad0c898e727 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:52.984076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac3b3-3307-4fe6-8ada-aad0c898e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:53.027389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-b0bac3b3-3307-4fe6-8ada-aad0c898e727 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:53.030658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b0bac3b3-3307-4fe6-8ada-aad0c898e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:53.243039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:53.251357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 831/1652] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:53.251916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:53.252004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067641c-f983-4161-8b48-d9209299a150 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:53.252594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067641c-f983-4161-8b48-d9209299a150 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:53.252928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067641c-f983-4161-8b48-d9209299a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:53.272219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be91ce-5300-43ba-9b9c-dd7e30e8314f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:53.308498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1067641c-f983-4161-8b48-d9209299a150 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:53.310050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1067641c-f983-4161-8b48-d9209299a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:53.313823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:53.314498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:53.314782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:53.315065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] PATH_INFO: `/v3/users/d3174cfa2b474a7b90d337a16fe6e522` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:53.320116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3174cfa2b474a7b90d337a16fe6e522)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:53.322901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:53.349012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 832/1653] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:53.350131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067641c-f983-4161-8b48-d9209299a150 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:53.379825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067641c-f983-4161-8b48-d9209299a150 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:53.392298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:53.395630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:53.395630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:53.395630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:53.395630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:53.396597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:53.398456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:53.438281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 833/1654] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:53.442928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:53.502547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033e4f4-8a7f-4599-bf5c-351927f0795d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:53.523486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:53.524137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:53.524465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:53.524783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:53.525543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:53.551336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:53.717658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 834/1655] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:53 2022] POST /identity/v3/projects => generated 363 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:53.722054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:53.752656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3174cfa2b474a7b90d337a16fe6e522'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:53.803158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:53.853945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3174cfa2b474a7b90d337a16fe6e522'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:53.855007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3174cfa2b474a7b90d337a16fe6e522'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:53.889494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3174cfa2b474a7b90d337a16fe6e522'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:53.925778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3174cfa2b474a7b90d337a16fe6e522'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:53.930076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab0a3e-4756-499f-9bdf-6a628a20e2cd admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:53.947006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c1f22-bda2-45ed-8188-a40a7db87266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:53.947006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c1f22-bda2-45ed-8188-a40a7db87266 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:53.947006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c1f22-bda2-45ed-8188-a40a7db87266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:53.947006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c1f22-bda2-45ed-8188-a40a7db87266 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:53.947548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c1f22-bda2-45ed-8188-a40a7db87266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:53.949728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c1f22-bda2-45ed-8188-a40a7db87266 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:53.988731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 822/1656] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:34:53 2022] DELETE /identity/v3/users/d3174cfa2b474a7b90d337a16fe6e522 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:53.992751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:54.002496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.100654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 835/1657] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:53 2022] POST /identity/v3/users => generated 282 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:54.102965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:54.104084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:54.104665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:54.132804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fe306-bb59-428d-9cc8-5ea99b43a257 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:54.139891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:54.140816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:54.141196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:54.145695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] PATH_INFO: `/v3/projects/751d12acbdb540539718f10f4edf9213` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:54.145695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=751d12acbdb540539718f10f4edf9213)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:54.146307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:54.218322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:54.219579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.232806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 836/1658] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:54.233973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:54.235458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.264641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984525f4-6c17-479c-8ed9-bd4ab1e3399c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:54.282564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:54.283301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:54.283736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:54.284150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:54.284973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:54.286753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:54.310770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 837/1659] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:54.312160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:54.314718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.349302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751d12acbdb540539718f10f4edf9213', 'request_id': 'req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:54.372744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751d12acbdb540539718f10f4edf9213', 'request_id': 'req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:54.410837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751d12acbdb540539718f10f4edf9213', 'request_id': 'req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:54.461868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c2ee8-f63e-4893-9c7d-0cc0c57d320a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:54.543634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] The token cache is being invalidate because project 751d12acbdb540539718f10f4edf9213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:34:54.544260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:54.546310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:54.550919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 823/1660] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:34:53 2022] DELETE /identity/v3/projects/751d12acbdb540539718f10f4edf9213 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:54.552039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:54.558229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.647505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:54.648627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:54.648901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:54.649166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:54.655698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.657024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:54.738786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ad3ba-aa17-4e79-bcf2-21c42b2587f3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:54.779606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:54.779606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:54.779606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:54.779606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:54.780554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:54.788365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:54.828688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 824/1661] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:54.829625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:54.840168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:54.892438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:54.896747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:54.914693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38f729-6696-4e94-a115-4ee0abe57555 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:54.928246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:54.928852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:54.929136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:54.929413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] PATH_INFO: `/v3/users/6e157fdc3bcd4ee88271ccc372d42660` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:54.932181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e157fdc3bcd4ee88271ccc372d42660)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:54.934309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:54.952561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 838/1662] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:54.959014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:55.003493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6eb53c-c8ef-4f30-8f68-a7cce25d6300 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:55.021225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcf206-1bd4-432e-8bde-59459bf585ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:55.021914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcf206-1bd4-432e-8bde-59459bf585ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:55.022191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcf206-1bd4-432e-8bde-59459bf585ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:55.022467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcf206-1bd4-432e-8bde-59459bf585ea admin admin] PATH_INFO: `/v3/projects/6cc81b8129af42458316cc31e39b9a1b/users/5ff75921243045cea42fd5e6496da63e/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:55.048832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcf206-1bd4-432e-8bde-59459bf585ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc81b8129af42458316cc31e39b9a1b, user_id=5ff75921243045cea42fd5e6496da63e, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:55.051749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcf206-1bd4-432e-8bde-59459bf585ea admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:55.190691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:55.206496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 839/1663] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:54 2022] PUT /identity/v3/projects/6cc81b8129af42458316cc31e39b9a1b/users/5ff75921243045cea42fd5e6496da63e/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:55.221632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:55.222338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:55.222618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:55.233112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e157fdc3bcd4ee88271ccc372d42660'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:55.301460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] MFA Rules not processed for user `5ff75921243045cea42fd5e6496da63e`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:55.306381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:55.341175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e157fdc3bcd4ee88271ccc372d42660'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:55.345462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e157fdc3bcd4ee88271ccc372d42660'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:55.362921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e157fdc3bcd4ee88271ccc372d42660'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:55.389739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e157fdc3bcd4ee88271ccc372d42660'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:55.420014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 825/1664] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:34:54 2022] DELETE /identity/v3/users/6e157fdc3bcd4ee88271ccc372d42660 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:55.466607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:55.472429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:55.476851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 840/1665] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:55.587938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305f102-ca8e-47b8-92ec-7ef5c313cf84 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:55.596924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:55.597815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:55.598258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:55.598720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:55.599507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:55.602106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:55.732932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 826/1666] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:55 2022] POST /identity/v3/projects => generated 363 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:55.737693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:55.891133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789e7a6-fc00-4f3b-8ac4-46e6d83cbacf None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:56.026873 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:56.027815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.028305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.028793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.029594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:56.034072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:56.148178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 841/1667] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:55 2022] POST /identity/v3/users => generated 288 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:56.153171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:56.167110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dcfe6-b41c-481f-be37-dd69d5709720 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:56.189175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:56.190195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.190713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.194649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.195723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:56.198096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:56.211476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 827/1668] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:56.216692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:56.234237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b21c53-4b9c-4ab5-b49d-a01c4a5d8d2c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:56.240398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:56.241434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.241931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.242553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] PATH_INFO: `/v3/projects/852084a85cc94fbe8d07fce370a9af54/users/df4fe0b9cc8f4676af1fa5ddaa62cae3/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.246344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:56.271038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852084a85cc94fbe8d07fce370a9af54, user_id=df4fe0b9cc8f4676af1fa5ddaa62cae3, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:56.275784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:56.398661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:56.407231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 842/1669] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:56 2022] PUT /identity/v3/projects/852084a85cc94fbe8d07fce370a9af54/users/df4fe0b9cc8f4676af1fa5ddaa62cae3/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:56.411001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:56.433085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaaf9d-158e-4d6c-8210-e64cb3530f25 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:56.562765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ef070-5c61-47b0-aa02-178249eaa487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:56.563402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ef070-5c61-47b0-aa02-178249eaa487 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.563685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ef070-5c61-47b0-aa02-178249eaa487 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.563961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ef070-5c61-47b0-aa02-178249eaa487 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.565047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ef070-5c61-47b0-aa02-178249eaa487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:56.566757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ef070-5c61-47b0-aa02-178249eaa487 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:56.589962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 828/1670] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:34:56.594634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:56.611467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985990c6-0eab-46b2-b7ba-9e0c808ecac9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:56.612469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2201e38-0c4d-484e-81a6-8e7eb34572a4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.613264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2201e38-0c4d-484e-81a6-8e7eb34572a4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.613733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2201e38-0c4d-484e-81a6-8e7eb34572a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.619078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53404-2032-44ff-a0e2-62565caf8a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:56.621474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53404-2032-44ff-a0e2-62565caf8a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.623206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53404-2032-44ff-a0e2-62565caf8a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.624444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53404-2032-44ff-a0e2-62565caf8a4e admin admin] PATH_INFO: `/v3/projects/852084a85cc94fbe8d07fce370a9af54/users/df4fe0b9cc8f4676af1fa5ddaa62cae3/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.633609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53404-2032-44ff-a0e2-62565caf8a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=852084a85cc94fbe8d07fce370a9af54, user_id=df4fe0b9cc8f4676af1fa5ddaa62cae3, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:56.637340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53404-2032-44ff-a0e2-62565caf8a4e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:56.679973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d2201e38-0c4d-484e-81a6-8e7eb34572a4 None None] MFA Rules not processed for user `703b8e63fa4449dfb85de594a8d2e5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:56.683018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d2201e38-0c4d-484e-81a6-8e7eb34572a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:56.708099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 843/1671] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:56 2022] PUT /identity/v3/projects/852084a85cc94fbe8d07fce370a9af54/users/df4fe0b9cc8f4676af1fa5ddaa62cae3/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:56.719429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.719708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.720146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.797999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] MFA Rules not processed for user `df4fe0b9cc8f4676af1fa5ddaa62cae3`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:56.799904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:56.857333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 829/1672] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:34:56.872281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:56.872861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:56.873361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:56.920529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] MFA Rules not processed for user `703b8e63fa4449dfb85de594a8d2e5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:56.921868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:56.943380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 830/1673] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:34:56.957274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:56.959445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:57.062708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 844/1674] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:57.080712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:57.083315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:57.157652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc4313-da8e-47f9-b0b1-4e3be6a3baf1 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:57.182539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:57.183326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.184504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.184788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:57.223080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:57.223080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:57.235810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 831/1675] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:57.237925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:57.239453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:57.262475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf653de-d643-4eb6-8873-fbfcc38d15de None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:57.279498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:57.280838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.281110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.281379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:57.298706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:57.300147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:57.335876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 845/1676] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:57.386023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2404a-091e-443c-b35d-d71249dba456 service nova] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:57.395020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:57.395728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.396677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.396677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] PATH_INFO: `/v3/projects/e931d084c75b4dd3871922e92e278f88` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:57.397644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e931d084c75b4dd3871922e92e278f88)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:57.400733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:57.489490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:57.495750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:57.531781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344ca19-da45-4b7a-b934-149e4051fc0e service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:57.542813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:57.545299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.551653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.551653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] PATH_INFO: `/v3/projects/6da1377b83bc48c28706ac7d0b9e6360` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:57.553435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da1377b83bc48c28706ac7d0b9e6360)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:57.556449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:57.631024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e931d084c75b4dd3871922e92e278f88', 'request_id': 'req-a6dbadbd-93e4-4b25-b827-ad910d032489'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.655190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e931d084c75b4dd3871922e92e278f88', 'request_id': 'req-a6dbadbd-93e4-4b25-b827-ad910d032489'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.676449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e931d084c75b4dd3871922e92e278f88', 'request_id': 'req-a6dbadbd-93e4-4b25-b827-ad910d032489'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.753224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:57.828463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] The token cache is being invalidate because project e931d084c75b4dd3871922e92e278f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:34:57.834269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.840648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a6dbadbd-93e4-4b25-b827-ad910d032489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.847162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 832/1677] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:34:57 2022] DELETE /identity/v3/projects/e931d084c75b4dd3871922e92e278f88 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:57.854414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f639e-8e55-4d51-9973-26417d246824 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.854414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f639e-8e55-4d51-9973-26417d246824 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.854414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f639e-8e55-4d51-9973-26417d246824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:57.884977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d01f639e-8e55-4d51-9973-26417d246824 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:57.886269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d01f639e-8e55-4d51-9973-26417d246824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:57.899185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da1377b83bc48c28706ac7d0b9e6360', 'request_id': 'req-a39cffcd-7e68-4863-9865-2f0abcec220f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.907181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:57.985822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 833/1678] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:57.986966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da1377b83bc48c28706ac7d0b9e6360', 'request_id': 'req-a39cffcd-7e68-4863-9865-2f0abcec220f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:57.987888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9465fc-6fdc-491b-8db0-74e2ca13e416 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.988347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9465fc-6fdc-491b-8db0-74e2ca13e416 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.989502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9465fc-6fdc-491b-8db0-74e2ca13e416 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:57.991703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 834/1679] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 00:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:34:57.994602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017212e8-2383-4334-b9b6-840393696649 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:57.995048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017212e8-2383-4334-b9b6-840393696649 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:57.996239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017212e8-2383-4334-b9b6-840393696649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.017394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da1377b83bc48c28706ac7d0b9e6360', 'request_id': 'req-a39cffcd-7e68-4863-9865-2f0abcec220f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:58.097246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-017212e8-2383-4334-b9b6-840393696649 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:58.098913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-017212e8-2383-4334-b9b6-840393696649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:58.099146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] The token cache is being invalidate because project 6da1377b83bc48c28706ac7d0b9e6360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:34:58.100452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:58.105751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a39cffcd-7e68-4863-9865-2f0abcec220f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:58.115790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 846/1680] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:34:57 2022] DELETE /identity/v3/projects/6da1377b83bc48c28706ac7d0b9e6360 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:58.116925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.119604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.120104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.127098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 835/1681] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:58.131265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.131579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.131854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.195854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:58.200877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:58.230418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:58.234237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:58.241636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 836/1682] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:58.242931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.269753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496affd0-b2ba-43ce-be2e-bd62e9b79cef None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:58.287363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:58.288221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.288518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.288789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.291438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:58.291946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:58.299127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4e6f9-33f9-4884-aa08-b9fdd7396a8e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:58.343317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 837/1683] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:58.347244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.347560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.347841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.379585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 847/1684] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:58.381113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.417370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561693ed-d102-478f-8f36-4e422f710a83 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:58.424886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:58.427599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:58.427599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.427599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.427599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.428028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:58.428173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:58.430420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:58.454829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 838/1685] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:58.463104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.499274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac40ef-4b6f-4611-b679-b55f97afc030 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:58.512855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:58.513917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.515488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.516694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.518188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:34:58.518799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:58.520852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:58.541514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 839/1686] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:34:58.542488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.546454 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:58.565869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f71e1-5f0a-4f25-b267-8506ff010890 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:58.579757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:58.581408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.581690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.581962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.610404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:58.612248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:58.619714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 848/1687] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/projects => generated 371 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:58.623043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.646926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 840/1688] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:58.647779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.687038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0b34-7ea4-4ad5-9d0b-7c252109c488 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:58.808159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f80c44-1561-4817-a438-ce32fb1594f6 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:58.825434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:58.826074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.826347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.827318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.828481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:58.831674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:58.844446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:58.845350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:58.845777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:58.846204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:58.847000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:58.849167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:58.902946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:58.940446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 849/1689] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/projects => generated 391 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:58.944569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.950541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:58.968383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 841/1690] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/users => generated 286 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:58.974014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:58.998312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91f853-dfa7-4df9-a767-bf82974da20d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.060058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73910a0-7298-44de-8891-9a1c5828cea4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.084441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.086171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.087427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.088308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] PATH_INFO: `/v3/users/5ff75921243045cea42fd5e6496da63e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.092342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff75921243045cea42fd5e6496da63e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.100238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.154889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.155446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.155779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.155987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.156656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.158599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.286154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 842/1691] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:34:58 2022] POST /identity/v3/users => generated 296 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:34:59.288450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:59.332076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b4b17-fb79-4ddc-9661-6a64dce90aa8 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.348784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff75921243045cea42fd5e6496da63e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:59.350905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.351544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.351823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.352089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.352885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.354524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.398520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff75921243045cea42fd5e6496da63e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:59.400696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 843/1692] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:59.402399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:59.432794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff75921243045cea42fd5e6496da63e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:59.442810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd55b87-28fa-4536-88ac-d95338c470b2 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.461414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.462576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.462855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.463144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.464075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.465968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.495674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 844/1693] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:59.496746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:59.529159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e4ac-415c-43b7-acb4-2efa61676362 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.534628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a39fd-de0e-404d-a5da-62f1402268bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.535153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a39fd-de0e-404d-a5da-62f1402268bd admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.535432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a39fd-de0e-404d-a5da-62f1402268bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.535705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a39fd-de0e-404d-a5da-62f1402268bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.538676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a39fd-de0e-404d-a5da-62f1402268bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.538854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a39fd-de0e-404d-a5da-62f1402268bd admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.558678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff75921243045cea42fd5e6496da63e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:59.559356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff75921243045cea42fd5e6496da63e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:34:59.567426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 850/1694] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:34:58 2022] DELETE /identity/v3/users/5ff75921243045cea42fd5e6496da63e => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:34:59.569190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:59.572149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:59.591538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 845/1695] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:59.595344 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.595344 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.595344 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.670573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:59.674720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:59.795631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec4726-611b-41c1-b63e-39fe93c6be19 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.803928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ef668-dfaa-4398-956f-616c13bfc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.804887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ef668-dfaa-4398-956f-616c13bfc0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.805377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ef668-dfaa-4398-956f-616c13bfc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.806612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ef668-dfaa-4398-956f-616c13bfc0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.808303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ef668-dfaa-4398-956f-616c13bfc0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.810964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ef668-dfaa-4398-956f-616c13bfc0f0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.846064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 846/1696] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:34:59.847268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:59.857184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 851/1697] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:34:59.857573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:59.857731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.857731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.857731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.871662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e5d3-6df2-4097-a51b-0c9e28a44bd8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:59.876245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:59.878029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:59.880685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:59.883478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:59.883478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] PATH_INFO: `/v3/projects/2a0cd7c4b8df4b28b076d9dbbb788d5b/users/a29ce09a283d44e88c753bf71a725819/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:59.917707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0cd7c4b8df4b28b076d9dbbb788d5b, user_id=a29ce09a283d44e88c753bf71a725819, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:59.928841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:59.947243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] MFA Rules not processed for user `77d23c99308c414f939761b07d329e03`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:34:59.947243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.062997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 847/1698] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:34:59 2022] PUT /identity/v3/projects/2a0cd7c4b8df4b28b076d9dbbb788d5b/users/a29ce09a283d44e88c753bf71a725819/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:00.064772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:00.070040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.106347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 852/1699] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:00.106347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:00.119809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19823b-47c5-4b3f-b3f8-91ead6cdb84b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:00.202501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3146d-c216-44b8-92f3-bac456859cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:00.203257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3146d-c216-44b8-92f3-bac456859cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.203477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3146d-c216-44b8-92f3-bac456859cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.203756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3146d-c216-44b8-92f3-bac456859cdb admin admin] PATH_INFO: `/v3/projects/db55b01271fe492db5eb8f93f3a32dd3/users/219bdb6a2c0c48bfb4a9f0a47a33c956/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.233327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3146d-c216-44b8-92f3-bac456859cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=db55b01271fe492db5eb8f93f3a32dd3, user_id=219bdb6a2c0c48bfb4a9f0a47a33c956, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:00.236495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3146d-c216-44b8-92f3-bac456859cdb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:00.247547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc299dab-51da-46dc-92df-8c3c9d965548 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:00.257398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:00.258276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.258788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.259260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.262329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.295855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:00.297824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:00.326673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 848/1700] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:00.327707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:00.330626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.395995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 853/1701] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:00 2022] PUT /identity/v3/projects/db55b01271fe492db5eb8f93f3a32dd3/users/219bdb6a2c0c48bfb4a9f0a47a33c956/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:00.399177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabbaa7-ddea-4103-845a-5fe8a19ce434 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.402231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabbaa7-ddea-4103-845a-5fe8a19ce434 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.403626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabbaa7-ddea-4103-845a-5fe8a19ce434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.472050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-deabbaa7-ddea-4103-845a-5fe8a19ce434 None None] MFA Rules not processed for user `a29ce09a283d44e88c753bf71a725819`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:00.473624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-deabbaa7-ddea-4103-845a-5fe8a19ce434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.479002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1de8a-a9ab-46fd-ab8d-f64c3472c3d0 service placement] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:00.626239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 854/1702] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:00.626239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563a1bd-b633-4acf-b1d9-6bd5c15cd0dd None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.626704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563a1bd-b633-4acf-b1d9-6bd5c15cd0dd None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.627169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563a1bd-b633-4acf-b1d9-6bd5c15cd0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.634707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:00.635687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.636082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.636647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.657364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c563a1bd-b633-4acf-b1d9-6bd5c15cd0dd None None] MFA Rules not processed for user `77d23c99308c414f939761b07d329e03`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:00.658907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c563a1bd-b633-4acf-b1d9-6bd5c15cd0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.777987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 855/1703] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:00.777987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.777987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.777987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.829984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:00.837379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:00.846756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] MFA Rules not processed for user `219bdb6a2c0c48bfb4a9f0a47a33c956`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:00.848373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.856163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 849/1704] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:00.857050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:00.876570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01578d-0020-4623-ac6d-0bcf123be7d1 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:00.890692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:00.891293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:00.891592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:00.891842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:00.892497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:00.898026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:00.955403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 856/1705] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:00.957003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:00.958995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:00.996648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 850/1706] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:00 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:00.998929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.017014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:01.080565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01f86c-759b-49bb-9c1b-14969cdd1c05 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.139620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc08787-aec4-47eb-afcf-0e3b64e300da admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.155053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.155740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.156019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.156294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.157122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.159830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.175976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.176727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.179191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.179191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.288318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 851/1707] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:00 2022] POST /identity/v3/projects => generated 391 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:01.289208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.328228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.333017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.348644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 857/1708] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:01.351949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.364828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd43d-5f30-4f62-aa95-23f82096107f service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.463710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ed67d-c871-48ac-9f28-47ee3f7a9729 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.493198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.494281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.494801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.495284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.496193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.499015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.623519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 852/1709] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:01 2022] POST /identity/v3/users => generated 292 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:01.628842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.656368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f285a-9bbb-4ad3-9884-12fbf31e9f60 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.667487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.667911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.669592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.670078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.670232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.670624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.671393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.671844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.672728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.675107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.677027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.681625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.718576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 853/1710] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:01.723359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.748355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742a5d4-fbf0-4a8b-a28c-3241140afc09 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.764304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.765017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.765555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.766029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] PATH_INFO: `/v3/projects/df4c8b67d5a54f67b2720eec40c9f028/users/1ecdc02504ae4c33aa2ec0f8b2a06d8e/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.799792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4c8b67d5a54f67b2720eec40c9f028, user_id=1ecdc02504ae4c33aa2ec0f8b2a06d8e, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.801784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 858/1711] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:01 2022] POST /identity/v3/users => generated 302 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:01.805341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.809252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.833442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a676-fe13-4404-b5ac-bebcc4333fa8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:01.847260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:01.850012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:01.850306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:01.850594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:01.869496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:01.874118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:01.897288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 854/1712] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:01 2022] PUT /identity/v3/projects/df4c8b67d5a54f67b2720eec40c9f028/users/1ecdc02504ae4c33aa2ec0f8b2a06d8e/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:01.898221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.919311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 859/1713] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:01.920196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:01.951694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe7b9c-29fe-40eb-9674-8a2540ee752c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:02.049056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c02bce-4671-430e-b87e-6b9969689702 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:02.067320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:02.067320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.067320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.067320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.067831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:02.068074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:02.108792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 855/1714] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:02.111456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:02.136062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd06798-b88c-4d8e-ab5f-15e29b5ae709 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:02.136284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:02.137083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:02.137838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.138196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.138546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] PATH_INFO: `/v3/projects/761e9d33c9db4554a47c8d710babaab9/users/40284cd3d4ff4883a7affea0583de7fb/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.161569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b6b84-adea-46e6-94fa-3adbda85955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:02.162572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6b84-adea-46e6-94fa-3adbda85955a admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.163052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6b84-adea-46e6-94fa-3adbda85955a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.163521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6b84-adea-46e6-94fa-3adbda85955a admin admin] PATH_INFO: `/v3/projects/df4c8b67d5a54f67b2720eec40c9f028/users/1ecdc02504ae4c33aa2ec0f8b2a06d8e/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.171158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b6b84-adea-46e6-94fa-3adbda85955a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4c8b67d5a54f67b2720eec40c9f028, user_id=1ecdc02504ae4c33aa2ec0f8b2a06d8e, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:02.173937 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b6b84-adea-46e6-94fa-3adbda85955a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:02.187595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761e9d33c9db4554a47c8d710babaab9, user_id=40284cd3d4ff4883a7affea0583de7fb, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:02.190581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:02.222542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 856/1715] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:02 2022] PUT /identity/v3/projects/df4c8b67d5a54f67b2720eec40c9f028/users/1ecdc02504ae4c33aa2ec0f8b2a06d8e/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:02.233984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.234494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.249709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.270847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 860/1716] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:01 2022] PUT /identity/v3/projects/761e9d33c9db4554a47c8d710babaab9/users/40284cd3d4ff4883a7affea0583de7fb/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:02.274730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:02.374425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] MFA Rules not processed for user `1ecdc02504ae4c33aa2ec0f8b2a06d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:02.375775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:02.557603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:02.586998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aba0dd-2dea-4479-b831-8df865827a54 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:02.601299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:02.601930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.604421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.604904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.606619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:02.612474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:02.641230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 861/1717] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:02.649882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:02.682235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53230294-6a25-416f-9144-6e56b6dae5f2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:02.717955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c6c02-3c28-4172-afdb-ba54f8d1eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:02.718365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 857/1718] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:02.719444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c6c02-3c28-4172-afdb-ba54f8d1eed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.719444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c6c02-3c28-4172-afdb-ba54f8d1eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.719444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c6c02-3c28-4172-afdb-ba54f8d1eed5 admin admin] PATH_INFO: `/v3/projects/761e9d33c9db4554a47c8d710babaab9/users/40284cd3d4ff4883a7affea0583de7fb/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.733048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:02.737433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:02.740555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c6c02-3c28-4172-afdb-ba54f8d1eed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761e9d33c9db4554a47c8d710babaab9, user_id=40284cd3d4ff4883a7affea0583de7fb, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:02.751059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c6c02-3c28-4172-afdb-ba54f8d1eed5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:02.799437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 862/1719] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:02 2022] PUT /identity/v3/projects/761e9d33c9db4554a47c8d710babaab9/users/40284cd3d4ff4883a7affea0583de7fb/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:02.809775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:02.810488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:02.811002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:02.911778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] MFA Rules not processed for user `40284cd3d4ff4883a7affea0583de7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:02.914070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd833b-2bed-46cd-aceb-1772684aeaff None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:02.922204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:03.099292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:03.099940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:03.100223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:03.100536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:03.192420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 863/1720] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:03.385109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:03.386673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:03.428802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 858/1721] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:03.537088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:03.550374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cbcc5-f99f-4b4b-88de-0926c8675b97 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:03.557803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:03.559065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:03.559494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:03.559910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:03.577236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:03.579328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:03.611236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 864/1722] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:03.664356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:03.666795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:03.674791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:03.759327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:03.873608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8425a-7f9e-41b4-9ec7-f7df2f1e13ff service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:03.898644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:03.900836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:03.901123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:03.901757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] PATH_INFO: `/v3/projects/6cc81b8129af42458316cc31e39b9a1b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:03.902613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc81b8129af42458316cc31e39b9a1b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:03.904581 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:03.915953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:03.917853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:03.947921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21a207-31ea-4f8f-a870-30c4572fc232 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:03.965326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:03.965326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:03.965326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:03.965326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:04.101029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc81b8129af42458316cc31e39b9a1b', 'request_id': 'req-0989306e-3773-48ea-a369-98b79ccfb648'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.118542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc81b8129af42458316cc31e39b9a1b', 'request_id': 'req-0989306e-3773-48ea-a369-98b79ccfb648'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.123082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:04.134422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc81b8129af42458316cc31e39b9a1b', 'request_id': 'req-0989306e-3773-48ea-a369-98b79ccfb648'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.143420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:04.167538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 865/1723] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:04.268471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] The token cache is being invalidate because project 6cc81b8129af42458316cc31e39b9a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:04.268774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.271458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0989306e-3773-48ea-a369-98b79ccfb648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.273574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 859/1724] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:03 2022] DELETE /identity/v3/projects/6cc81b8129af42458316cc31e39b9a1b => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:04.277841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:04.295225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dc6a7-4dbc-4288-b7d5-d9ad9557710f service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:04.502619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:04.502619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:04.502962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:04.503419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] PATH_INFO: `/v3/users/df4fe0b9cc8f4676af1fa5ddaa62cae3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:04.508263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4fe0b9cc8f4676af1fa5ddaa62cae3)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:04.513583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:04.672340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811647b-ed38-4268-af46-55a37961d62d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:04.672860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811647b-ed38-4268-af46-55a37961d62d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:04.672941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811647b-ed38-4268-af46-55a37961d62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:04.728998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8811647b-ed38-4268-af46-55a37961d62d None None] MFA Rules not processed for user `77d23c99308c414f939761b07d329e03`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:04.730366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8811647b-ed38-4268-af46-55a37961d62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:04.867044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4fe0b9cc8f4676af1fa5ddaa62cae3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.877631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:04.900031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4fe0b9cc8f4676af1fa5ddaa62cae3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.921030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4fe0b9cc8f4676af1fa5ddaa62cae3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:04.942455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 860/1725] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:05.005633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4fe0b9cc8f4676af1fa5ddaa62cae3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:05.009816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4fe0b9cc8f4676af1fa5ddaa62cae3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:05.020170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 866/1726] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:04 2022] DELETE /identity/v3/users/df4fe0b9cc8f4676af1fa5ddaa62cae3 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:05.067263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:05.067551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:05.067827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:05.090359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:05.153400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] MFA Rules not processed for user `2a927401086b4a649919a6071b4a0fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:05.154680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:05.174764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:05.176546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:05.432524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59d0b9-825b-4d39-9dad-4773ea85fc66 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:05.446467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:05.448543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:05.449501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:05.451088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 861/1727] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:05.451411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] PATH_INFO: `/v3/users/a29ce09a283d44e88c753bf71a725819` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:05.455683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29ce09a283d44e88c753bf71a725819)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:05.457968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:05.467799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:05.469386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:05.665837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417fa1b-0a6c-471e-8133-f9fc6377b20d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:05.689297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11ec18-a272-4910-b66d-92db75cc8d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:05.690260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11ec18-a272-4910-b66d-92db75cc8d6d service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:05.690787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11ec18-a272-4910-b66d-92db75cc8d6d service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:05.691201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11ec18-a272-4910-b66d-92db75cc8d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:05.729100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11ec18-a272-4910-b66d-92db75cc8d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:05.733460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11ec18-a272-4910-b66d-92db75cc8d6d service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:05.752104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 862/1728] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:05.753883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:05.857456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29ce09a283d44e88c753bf71a725819'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:05.932779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29ce09a283d44e88c753bf71a725819'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:05.964791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29ce09a283d44e88c753bf71a725819'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:06.054007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:06.089263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29ce09a283d44e88c753bf71a725819'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:06.090083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29ce09a283d44e88c753bf71a725819'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:06.101301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 867/1729] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:05 2022] DELETE /identity/v3/users/a29ce09a283d44e88c753bf71a725819 => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:06.105778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:06.106148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:06.106431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:06.151287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:06.152617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:06.306112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 863/1730] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:06.410936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:06.417937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:06.651324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce83703-2848-43e8-beb4-2896bc32dc17 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:06.676742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb5a78-688f-4d05-9ff7-ab2479176cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:06.681061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5a78-688f-4d05-9ff7-ab2479176cfc service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:06.684887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5a78-688f-4d05-9ff7-ab2479176cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:06.685369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5a78-688f-4d05-9ff7-ab2479176cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:06.712605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb5a78-688f-4d05-9ff7-ab2479176cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:06.717699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb5a78-688f-4d05-9ff7-ab2479176cfc service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:06.773511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:06.775441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:06.811247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 868/1731] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:06.815259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c98fc-5216-4fd5-91bc-ff1980dc70da None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:06.832755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:06.849587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:06.852030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:06.857592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:06.857592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] PATH_INFO: `/v3/projects/852084a85cc94fbe8d07fce370a9af54` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:06.858219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] RBAC: Authorizing `identity:delete_project(project_id=852084a85cc94fbe8d07fce370a9af54)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:06.862846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:06.950185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955e2b-b1f3-47e1-bd82-a0ecb7374904 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:06.951684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955e2b-b1f3-47e1-bd82-a0ecb7374904 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:06.952310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955e2b-b1f3-47e1-bd82-a0ecb7374904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:07.064840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852084a85cc94fbe8d07fce370a9af54', 'request_id': 'req-92d515dc-c851-48f6-83db-50fba5feacae'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:07.076262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852084a85cc94fbe8d07fce370a9af54', 'request_id': 'req-92d515dc-c851-48f6-83db-50fba5feacae'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:07.079943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-81955e2b-b1f3-47e1-bd82-a0ecb7374904 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:07.081390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-81955e2b-b1f3-47e1-bd82-a0ecb7374904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:07.085985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852084a85cc94fbe8d07fce370a9af54', 'request_id': 'req-92d515dc-c851-48f6-83db-50fba5feacae'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:07.148054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:07.170101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] The token cache is being invalidate because project 852084a85cc94fbe8d07fce370a9af54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:07.170444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:07.171503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-92d515dc-c851-48f6-83db-50fba5feacae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:07.174097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 864/1732] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:06 2022] DELETE /identity/v3/projects/852084a85cc94fbe8d07fce370a9af54 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:07.178683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a2b5-937e-43d3-9763-81b4b8069928 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:07.179145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a2b5-937e-43d3-9763-81b4b8069928 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:07.179563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a2b5-937e-43d3-9763-81b4b8069928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:07.233240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3214a2b5-937e-43d3-9763-81b4b8069928 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:07.235228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3214a2b5-937e-43d3-9763-81b4b8069928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:07.238630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 869/1733] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:07.358805 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:07.359092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:07.359364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:07.448794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 865/1734] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:07.450441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:07.450735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:07.451057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:07.478741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] MFA Rules not processed for user `45422a67a78f4784abc0f18ad84778f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:07.498986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:07.572764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:07.574017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:07.593675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 866/1735] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:07.598256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:07.624933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639106a-b4d9-493c-aaec-5319fe0a71c9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:07.644975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:07.645598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:07.645875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:07.646140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:07.648881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:07.649197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:07.652648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:07.682941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 867/1736] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:07.686145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:07.712022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223ecf1-193a-4cf7-92fb-0407e0da5870 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:07.718427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:07.721510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:07.721510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:07.721510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:07.721510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:07.724084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:07.787729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 870/1737] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:07.802272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:07.806227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:07.929053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 868/1738] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:07 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:07.931135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:07.939677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:08.144414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bde2a-7051-43e9-b570-c79afbfa73ad None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:08.253954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebc073-73b8-4ec3-90d9-d61b1b1419d9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:08.283834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:08.285674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:08.287059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:08.287871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:08.298342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:08.301172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:08.304772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:08.308650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:08.461817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:08.473711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:08.499962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 869/1739] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:35:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:35:08.505453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:08.510524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:08.512162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:08.529737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:08.547360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 871/1740] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:08.770969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd915d3-af67-43e3-90e5-5246b97bb9f2 service swift] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:08.793294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:08.794100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:08.794384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:08.794671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:08.798057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:08.805571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:08.959388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 870/1741] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:08 2022] POST /identity/v3/users => generated 283 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:08.965795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:08.969767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:09.013781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6222d-46c0-4d2f-802b-d45c4eecbde4 service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:09.027188 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:09.034759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:09.036439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:09.037040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:09.038874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:09.046985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:09.115113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 872/1742] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:09.125493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:09.157720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3906bbb-e84b-4a68-8891-bc33b266afac admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:09.176962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2806d-bc05-4759-be5b-d65b4becc295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:09.177866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2806d-bc05-4759-be5b-d65b4becc295 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:09.178169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2806d-bc05-4759-be5b-d65b4becc295 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:09.178571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2806d-bc05-4759-be5b-d65b4becc295 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:09.179792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2806d-bc05-4759-be5b-d65b4becc295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:09.183127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2806d-bc05-4759-be5b-d65b4becc295 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:09.221123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 871/1743] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:09.224871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:09.259398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bd7b2-3dae-4378-ae80-ef255d290add admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:09.281509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:09.281509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:09.282157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:09.282396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] PATH_INFO: `/v3/projects/8b943671096f4cb28b4d46dd6c8503c9/users/ce5edb3ad4a24205b3be24914d58aa9a/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:09.326512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b943671096f4cb28b4d46dd6c8503c9, user_id=ce5edb3ad4a24205b3be24914d58aa9a, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:09.359648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:09.469495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 873/1744] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:09 2022] PUT /identity/v3/projects/8b943671096f4cb28b4d46dd6c8503c9/users/ce5edb3ad4a24205b3be24914d58aa9a/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:09.474647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:09.475383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:09.475597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:09.580085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] MFA Rules not processed for user `ce5edb3ad4a24205b3be24914d58aa9a`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:09.585945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:09.911333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:09.912024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 872/1745] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:09.950587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:09.955995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:10.026393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:10.275225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36006945-a434-40bc-8333-87f119912d55 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:10.280184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:10.295580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd29df-18a2-4612-98da-b66e4c5355d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:10.299197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd29df-18a2-4612-98da-b66e4c5355d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:10.299985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd29df-18a2-4612-98da-b66e4c5355d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:10.301133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd29df-18a2-4612-98da-b66e4c5355d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:10.302783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd29df-18a2-4612-98da-b66e4c5355d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:10.311093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd29df-18a2-4612-98da-b66e4c5355d9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:10.315135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313298c-2c61-422b-9721-f5fbdefd7e49 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:10.322525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:10.323739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:10.324174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:10.324632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] PATH_INFO: `/v3/projects/2a0cd7c4b8df4b28b076d9dbbb788d5b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:10.326088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a0cd7c4b8df4b28b076d9dbbb788d5b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:10.329814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:10.489297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 874/1746] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:09 2022] POST /identity/v3/projects => generated 367 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:10.490732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:10.492154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:10.496549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:10.525987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a0cd7c4b8df4b28b076d9dbbb788d5b', 'request_id': 'req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:10.555283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a0cd7c4b8df4b28b076d9dbbb788d5b', 'request_id': 'req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:10.571143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:10.575525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:10.582176 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a0cd7c4b8df4b28b076d9dbbb788d5b', 'request_id': 'req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:10.671634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] The token cache is being invalidate because project 2a0cd7c4b8df4b28b076d9dbbb788d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:10.672203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:10.673057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:10.680392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 873/1747] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:09 2022] DELETE /identity/v3/projects/2a0cd7c4b8df4b28b076d9dbbb788d5b => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:10.684684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:10.693519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:10.768986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 875/1748] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 31 00:35:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:10.770078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:10.777112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:10.876141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad70d5-38a0-45aa-adb2-25f93e63c83d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:10.891993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:10.892951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:10.893421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:10.894603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:10.895410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:10.904373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:11.006953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90cae6-f287-4ec4-8881-74d0f4e4fed5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:11.015221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:11.016319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:11.016747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:11.017098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] PATH_INFO: `/v3/users/1ecdc02504ae4c33aa2ec0f8b2a06d8e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:11.020536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ecdc02504ae4c33aa2ec0f8b2a06d8e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:11.023767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:11.041584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 874/1749] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:10 2022] POST /identity/v3/users => generated 290 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:11.048105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:11.094462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10568a-2c44-4303-b544-baa32339686a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:11.114395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:11.116772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:11.118951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:11.118951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:11.121012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:11.128103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:11.188424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 875/1750] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:11.194194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:11.206602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:11.253613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda38d7-bacf-4304-9d28-bb74549a9d2a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:11.280894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:11.280894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:11.281456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:11.283997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:11.318924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:11.323139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:11.333623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ecdc02504ae4c33aa2ec0f8b2a06d8e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:11.343317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 876/1751] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:11.344829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:11.358886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c8250-22f9-436e-8420-cd0534d9cc93 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:11.362495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ecdc02504ae4c33aa2ec0f8b2a06d8e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:11.365892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:11.367496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:11.368238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:11.368238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] PATH_INFO: `/v3/projects/2ef13e712abd4e11b61fd59642fe7608/users/5af044b562d9437c954cf5e73ad994b3/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:11.396200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ecdc02504ae4c33aa2ec0f8b2a06d8e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:11.396531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef13e712abd4e11b61fd59642fe7608, user_id=5af044b562d9437c954cf5e73ad994b3, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:11.400998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:11.498417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 877/1752] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:11 2022] PUT /identity/v3/projects/2ef13e712abd4e11b61fd59642fe7608/users/5af044b562d9437c954cf5e73ad994b3/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:11.504162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:11.518970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ecdc02504ae4c33aa2ec0f8b2a06d8e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:11.520821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ecdc02504ae4c33aa2ec0f8b2a06d8e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:11.531548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 876/1753] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:10 2022] DELETE /identity/v3/users/1ecdc02504ae4c33aa2ec0f8b2a06d8e => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:11.791429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a9644-b43f-4539-9615-cf98f7985e8b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:11.988380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:11.998761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d553326-eb52-4ec5-a20e-fa66a9194902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:11.999440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d553326-eb52-4ec5-a20e-fa66a9194902 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:11.999681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d553326-eb52-4ec5-a20e-fa66a9194902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:11.999951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d553326-eb52-4ec5-a20e-fa66a9194902 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:12.000672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d553326-eb52-4ec5-a20e-fa66a9194902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:12.002398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d553326-eb52-4ec5-a20e-fa66a9194902 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:12.026503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 878/1754] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:12.031003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:12.032474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:12.104791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd76a6-5ee8-4f70-babc-c56eef1e2fec admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:12.135471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:12.136227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:12.137124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:12.139594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] PATH_INFO: `/v3/projects/2ef13e712abd4e11b61fd59642fe7608/users/5af044b562d9437c954cf5e73ad994b3/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:12.145938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef13e712abd4e11b61fd59642fe7608, user_id=5af044b562d9437c954cf5e73ad994b3, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:12.151849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:12.181638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 877/1755] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:12 2022] PUT /identity/v3/projects/2ef13e712abd4e11b61fd59642fe7608/users/5af044b562d9437c954cf5e73ad994b3/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:12.192021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:12.192694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:12.192694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:12.262026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] MFA Rules not processed for user `5af044b562d9437c954cf5e73ad994b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:12.268785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:12.279341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:12.431727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:12.434411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:12.474524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 879/1756] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:12.498435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:12.498435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:12.564441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783c837-0ba2-4136-90d2-6446aeffc5ff admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:12.574878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:12.577802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:12.580094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:12.581047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] PATH_INFO: `/v3/users/219bdb6a2c0c48bfb4a9f0a47a33c956` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:12.584130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219bdb6a2c0c48bfb4a9f0a47a33c956)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:12.589664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:12.654516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110a06-587c-4aca-8a9e-2ff6ef19cb51 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:12.662367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:12.665156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:12.668400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:12.668899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:12.701525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:12.703206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:12.723174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 880/1757] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:12.926346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:12.944758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af23be8-298d-4381-a5ec-65bbdf7429bd service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:12.955281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:12.955979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:12.956266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:12.956591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:12.959966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:12.960783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219bdb6a2c0c48bfb4a9f0a47a33c956'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.023166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219bdb6a2c0c48bfb4a9f0a47a33c956'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.056984 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219bdb6a2c0c48bfb4a9f0a47a33c956'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.136528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219bdb6a2c0c48bfb4a9f0a47a33c956'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.137650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219bdb6a2c0c48bfb4a9f0a47a33c956'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.156575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 878/1758] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:12 2022] DELETE /identity/v3/users/219bdb6a2c0c48bfb4a9f0a47a33c956 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:13.157722 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:13.166291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:13.194982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:13.200499 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b1fcb-a803-4433-a057-a0c05e52e291 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:13.227690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 881/1759] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:13.232742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:13.232742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:13.232742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:13.271367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:13.293525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:13.294749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:13.386946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb65e75-083e-4452-8cc4-6875cdaaf3e2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:13.397290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:13.397740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:13.398019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:13.398293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] PATH_INFO: `/v3/projects/df4c8b67d5a54f67b2720eec40c9f028` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:13.399750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4c8b67d5a54f67b2720eec40c9f028)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:13.403387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:13.431718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 882/1760] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:13.440593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:13.443507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:13.446699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:13.599347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf706e3-b1b2-4642-bd60-09aec0618008 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:13.629712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4c8b67d5a54f67b2720eec40c9f028', 'request_id': 'req-02962ca8-55c8-4b0e-8695-67f6e51d77f0'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.723022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f6cd-2654-4b8c-98c7-bd9cd8520403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:13.724005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f6cd-2654-4b8c-98c7-bd9cd8520403 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:13.724728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f6cd-2654-4b8c-98c7-bd9cd8520403 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:13.725647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f6cd-2654-4b8c-98c7-bd9cd8520403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:13.736064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4c8b67d5a54f67b2720eec40c9f028', 'request_id': 'req-02962ca8-55c8-4b0e-8695-67f6e51d77f0'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.746964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4c8b67d5a54f67b2720eec40c9f028', 'request_id': 'req-02962ca8-55c8-4b0e-8695-67f6e51d77f0'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.818876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] The token cache is being invalidate because project df4c8b67d5a54f67b2720eec40c9f028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:13.819425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.819987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-02962ca8-55c8-4b0e-8695-67f6e51d77f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:13.822577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 879/1761] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:13 2022] DELETE /identity/v3/projects/df4c8b67d5a54f67b2720eec40c9f028 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:13.827303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:13.827601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:13.827872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:13.861658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f6cd-2654-4b8c-98c7-bd9cd8520403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:13.863523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f6cd-2654-4b8c-98c7-bd9cd8520403 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:13.886707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 883/1762] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:13.894360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:13.895629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:13.924467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 880/1763] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:13.929515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:13.930727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:13.931146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:13.994204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:13.997133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:14.031150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 884/1764] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:14.035802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:14.088441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113ab0-3cf9-46d3-a2f4-b7e832b6af18 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:14.117083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:14.125040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:14.125334 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:14.125606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:14.126302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:14.126602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:14.128302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:14.165572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 881/1765] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:14.172699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:14.191348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b621deb-ee30-463b-b1a1-28fe524253ca None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:14.207798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:14.207798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:14.210242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:14.210242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:14.210549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:14.218176 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:14.340583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 885/1766] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:14 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:14.344555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:14.359670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1d299-c1b6-4ae4-a5cf-727f46010a19 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:14.522355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:14.606041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:14.607057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:14.607400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:14.607686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:14.609423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:14.612540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:14.762077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 882/1767] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:14 2022] POST /identity/v3/users => generated 294 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:14.763513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:14.777440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93235-29f5-4b3f-89ad-9e8457899c41 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:14.783783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:14.784541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:14.784930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:14.785298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:14.786193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:14.788267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:14.828244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 886/1768] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:14.834441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:14.857626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cafbb9-1d4d-4379-ad32-18e4434989b8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:14.866827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0914cc-e37a-4982-8b66-78b9152e4cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:14.866827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0914cc-e37a-4982-8b66-78b9152e4cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:14.875589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0914cc-e37a-4982-8b66-78b9152e4cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:14.876101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0914cc-e37a-4982-8b66-78b9152e4cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:14.877884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0914cc-e37a-4982-8b66-78b9152e4cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:14.882362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0914cc-e37a-4982-8b66-78b9152e4cd4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:14.920626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 883/1769] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:14.922070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:14.965456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2990556-2f57-4a28-81c3-53bb74cafc4c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:15.031884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:15.031884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:15.031884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:15.031884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] PATH_INFO: `/v3/projects/783632082be6442b80cb407e72a36329/users/013431a0f5d946c68fbcb11624531c32/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:15.098370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=783632082be6442b80cb407e72a36329, user_id=013431a0f5d946c68fbcb11624531c32, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:15.133727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:15.675923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 887/1770] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:14 2022] PUT /identity/v3/projects/783632082be6442b80cb407e72a36329/users/013431a0f5d946c68fbcb11624531c32/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:15.691240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:15.691840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:15.692837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:15.715812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:15.717816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:15.771437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] MFA Rules not processed for user `013431a0f5d946c68fbcb11624531c32`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:15.781627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:15.912880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:15.988444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 884/1771] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:15.991690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:16.069462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c45f99-0f1a-46fa-adbd-58ca3bad5bc9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:16.103658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:16.104552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:16.104760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:16.105090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] PATH_INFO: `/v3/projects/db55b01271fe492db5eb8f93f3a32dd3` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:16.107533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db55b01271fe492db5eb8f93f3a32dd3)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:16.116799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:16.248050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ba517-0f77-4692-b394-04877dff0799 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:16.266429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:16.267878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:16.269449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:16.269924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:16.296044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:16.304017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:16.354440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db55b01271fe492db5eb8f93f3a32dd3', 'request_id': 'req-bb99b37a-16c7-4ac7-a747-471b8352ee0b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:16.389838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db55b01271fe492db5eb8f93f3a32dd3', 'request_id': 'req-bb99b37a-16c7-4ac7-a747-471b8352ee0b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:16.400787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db55b01271fe492db5eb8f93f3a32dd3', 'request_id': 'req-bb99b37a-16c7-4ac7-a747-471b8352ee0b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:16.498527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 885/1772] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:15 2022] POST /identity/v3/projects => generated 385 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:16.502287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:16.532816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] The token cache is being invalidate because project db55b01271fe492db5eb8f93f3a32dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:16.532816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:16.532816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:16.532816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 888/1773] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:15 2022] DELETE /identity/v3/projects/db55b01271fe492db5eb8f93f3a32dd3 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:16.535746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:16.537400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:16.732184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac38e2-ba7a-4623-9151-fee5b0c5e2e5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:16.750666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:16.751415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:16.751903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:16.752694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:16.754184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:16.761779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99b37a-16c7-4ac7-a747-471b8352ee0b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:16.770461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:16.774116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:16.776392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:16.776983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:16.777462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] PATH_INFO: `/v3/users/40284cd3d4ff4883a7affea0583de7fb` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:16.779315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40284cd3d4ff4883a7affea0583de7fb)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:16.782591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:16.817543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:16.946002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 886/1774] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:16 2022] POST /identity/v3/users => generated 299 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:16.947193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:16.953427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:16.987890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502159b6-2c84-480f-81fd-041ec0a60d81 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:17.003799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:17.004669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:17.005005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:17.005405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:17.006223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:17.014175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:17.097093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40284cd3d4ff4883a7affea0583de7fb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:17.101534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 887/1775] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:17.111281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:17.135802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73021a07-d4a4-480f-9df6-5322c6e1f932 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:17.155859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:17.157856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:17.158139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:17.158413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] PATH_INFO: `/v3/projects/ff35beac0ae244f99e07a6b2db069e78/users/e9085ce86e354166a00bddf8441c48ba/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:17.163954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:17.191737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40284cd3d4ff4883a7affea0583de7fb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:17.192440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40284cd3d4ff4883a7affea0583de7fb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:17.214952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff35beac0ae244f99e07a6b2db069e78, user_id=e9085ce86e354166a00bddf8441c48ba, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:17.218383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40284cd3d4ff4883a7affea0583de7fb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:17.219220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:17.246525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40284cd3d4ff4883a7affea0583de7fb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:17.284705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 889/1776] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:16 2022] DELETE /identity/v3/users/40284cd3d4ff4883a7affea0583de7fb => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:17.346082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 888/1777] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:17 2022] PUT /identity/v3/projects/ff35beac0ae244f99e07a6b2db069e78/users/e9085ce86e354166a00bddf8441c48ba/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:17.349933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:17.351406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:17.371241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:17.536859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e4672-4887-4573-bbb7-5dff39a35726 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:17.708119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71efe35-f140-4630-9ebf-4ef5abd28d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:17.709765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efe35-f140-4630-9ebf-4ef5abd28d0b admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:17.710207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efe35-f140-4630-9ebf-4ef5abd28d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:17.710504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71efe35-f140-4630-9ebf-4ef5abd28d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:17.711705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71efe35-f140-4630-9ebf-4ef5abd28d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:17.713571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71efe35-f140-4630-9ebf-4ef5abd28d0b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:17.743493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 890/1778] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:17.746677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:17.787544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494d44-258a-4e85-a5c8-20310c657706 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:17.810320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7825e-0904-4ba5-95b6-2309e1f9203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:17.811473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7825e-0904-4ba5-95b6-2309e1f9203a admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:17.811756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7825e-0904-4ba5-95b6-2309e1f9203a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:17.812236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7825e-0904-4ba5-95b6-2309e1f9203a admin admin] PATH_INFO: `/v3/projects/ff35beac0ae244f99e07a6b2db069e78/users/e9085ce86e354166a00bddf8441c48ba/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:17.814749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7825e-0904-4ba5-95b6-2309e1f9203a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff35beac0ae244f99e07a6b2db069e78, user_id=e9085ce86e354166a00bddf8441c48ba, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:17.818032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7825e-0904-4ba5-95b6-2309e1f9203a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:17.903857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 889/1779] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:17 2022] PUT /identity/v3/projects/ff35beac0ae244f99e07a6b2db069e78/users/e9085ce86e354166a00bddf8441c48ba/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:17.910451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811548d-2b96-4612-9744-b6aec475960f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:17.910451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811548d-2b96-4612-9744-b6aec475960f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:17.910451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811548d-2b96-4612-9744-b6aec475960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:18.015069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9811548d-2b96-4612-9744-b6aec475960f None None] MFA Rules not processed for user `e9085ce86e354166a00bddf8441c48ba`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:18.020666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9811548d-2b96-4612-9744-b6aec475960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:18.215871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:18.216379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:18.216490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:18.228130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 891/1780] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:18.239718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811548d-2b96-4612-9744-b6aec475960f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:18.242537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9811548d-2b96-4612-9744-b6aec475960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:18.269624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] MFA Rules not processed for user `2a927401086b4a649919a6071b4a0fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:18.270891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:18.424452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811548d-2b96-4612-9744-b6aec475960f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:18.428531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:18.429171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:18.430408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:18.430697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:18.431923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 890/1781] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:18.449682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:18.451526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:18.471426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 892/1782] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:18.542185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:18.765428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:18.806106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a64caa-83b2-4af4-b889-ce1312388e79 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:18.815303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:18.816478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:18.816908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:18.817774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] PATH_INFO: `/v3/users/013431a0f5d946c68fbcb11624531c32` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:18.830283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=013431a0f5d946c68fbcb11624531c32)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:18.834042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:18.931565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:18.933917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:18.967695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a6f90-7035-4f3f-8f58-70ac20a508dd service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:18.990616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:18.991322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:18.991575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:18.991892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] PATH_INFO: `/v3/projects/761e9d33c9db4554a47c8d710babaab9` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.001264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=761e9d33c9db4554a47c8d710babaab9)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:19.003488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:19.109816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013431a0f5d946c68fbcb11624531c32'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.113782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:19.144954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013431a0f5d946c68fbcb11624531c32'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.167239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013431a0f5d946c68fbcb11624531c32'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.195805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761e9d33c9db4554a47c8d710babaab9', 'request_id': 'req-ae794727-a1d0-4fa3-8140-5453dedce078'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.213466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761e9d33c9db4554a47c8d710babaab9', 'request_id': 'req-ae794727-a1d0-4fa3-8140-5453dedce078'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.226861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761e9d33c9db4554a47c8d710babaab9', 'request_id': 'req-ae794727-a1d0-4fa3-8140-5453dedce078'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.249337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013431a0f5d946c68fbcb11624531c32'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.251242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c21b1ce9-72e9-42dd-8f25-bd5ebbed2ac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013431a0f5d946c68fbcb11624531c32'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.290576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 891/1783] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:18 2022] DELETE /identity/v3/users/013431a0f5d946c68fbcb11624531c32 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:19.296189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946622-398c-49d5-8cba-45c828790e68 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.296672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946622-398c-49d5-8cba-45c828790e68 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.297006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946622-398c-49d5-8cba-45c828790e68 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.299010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 892/1784] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:35:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:35:19.300050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77260ccb-697d-442f-9eea-7a44fedfda49 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.309989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77260ccb-697d-442f-9eea-7a44fedfda49 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.310355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77260ccb-697d-442f-9eea-7a44fedfda49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.345893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] The token cache is being invalidate because project 761e9d33c9db4554a47c8d710babaab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:19.346261 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.356264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ae794727-a1d0-4fa3-8140-5453dedce078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:19.358646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 893/1785] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:18 2022] DELETE /identity/v3/projects/761e9d33c9db4554a47c8d710babaab9 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:19.360780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423491da-5491-4420-b033-d863522b6c31 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.361299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423491da-5491-4420-b033-d863522b6c31 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.361810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423491da-5491-4420-b033-d863522b6c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.370996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-77260ccb-697d-442f-9eea-7a44fedfda49 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:19.372602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-77260ccb-697d-442f-9eea-7a44fedfda49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:19.420122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-423491da-5491-4420-b033-d863522b6c31 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:19.421604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-423491da-5491-4420-b033-d863522b6c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:19.552349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 893/1786] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:19.559428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.560204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.560829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.577690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 894/1787] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:19.579594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423491da-5491-4420-b033-d863522b6c31 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:19.581433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-423491da-5491-4420-b033-d863522b6c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:19.595971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:19.597628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:19.636557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 894/1788] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:19.636557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:19.636557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:19.686478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d2a67-8498-4934-8cda-14d4b8e12b2f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:19.688764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fc8df-fe88-4253-8a02-dc5f62d48151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:19.689594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fc8df-fe88-4253-8a02-dc5f62d48151 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.689972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fc8df-fe88-4253-8a02-dc5f62d48151 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.690215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fc8df-fe88-4253-8a02-dc5f62d48151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.695663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423491da-5491-4420-b033-d863522b6c31 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:19.702960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:19.704542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.705308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.705776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.707596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fc8df-fe88-4253-8a02-dc5f62d48151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:19.710049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fc8df-fe88-4253-8a02-dc5f62d48151 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:19.723772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 895/1789] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:19.726010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.727118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.730838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:19.732766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.733002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:19.765789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:19.767759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 895/1790] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:19.781085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:19.823449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 896/1791] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:19.829027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:19.846686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c18d47-f51a-4324-842c-469fa68cda49 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:19.856737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:19.860615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.860908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.861275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.862154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:19.862466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:19.869714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5d1d6-30bd-4973-899d-48b70dc51d06 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:19.908117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 896/1792] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:19.914879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:19.936131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3f3df-c698-4852-bc4c-ced98cf6437e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:19.954778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:19.955422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:19.958652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:19.959179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:19.959948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:19.964568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:20.049480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.061148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.061148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.082328 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 897/1793] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:19 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:20.087688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:20.110762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] MFA Rules not processed for user `45422a67a78f4784abc0f18ad84778f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:20.116481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:20.249278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dd7dd-101d-4cb5-a58d-4621577ff313 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:20.256180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:20.256993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.257256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.258164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.259930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:20.261917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 897/1794] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:20.262883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:20.363891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 898/1795] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:20 2022] POST /identity/v3/users => generated 284 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:20.368631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:20.389235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042a2a6-a1c6-49ff-994b-8a31855669e5 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:20.413791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:20.417857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.418376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.421779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.424017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:20.429151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:20.463691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 898/1796] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:20.465439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:20.487609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c9582-d525-4cc2-a867-f7a8362b0bd8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:20.509291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3780372-39ea-48f6-a6e2-d1ee24f083dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:20.513087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3780372-39ea-48f6-a6e2-d1ee24f083dd admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.515859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3780372-39ea-48f6-a6e2-d1ee24f083dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.516191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3780372-39ea-48f6-a6e2-d1ee24f083dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.517369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3780372-39ea-48f6-a6e2-d1ee24f083dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:20.520982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3780372-39ea-48f6-a6e2-d1ee24f083dd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:20.534528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 899/1797] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:20.539819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:20.571102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499dc42-720f-4033-a211-fbafd6eb99ca admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:20.591654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:20.598539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.598539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.598539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] PATH_INFO: `/v3/projects/1360f38a67d442af95a5ce86ef699d6f/users/338272c0439044b3936d41d25789314e/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.619620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1360f38a67d442af95a5ce86ef699d6f, user_id=338272c0439044b3936d41d25789314e, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:20.622928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:20.731196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 899/1798] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:20 2022] PUT /identity/v3/projects/1360f38a67d442af95a5ce86ef699d6f/users/338272c0439044b3936d41d25789314e/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:20.747210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.747830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.748680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.787731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:20.794955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:20.798859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] MFA Rules not processed for user `338272c0439044b3936d41d25789314e`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:20.800492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:20.884486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 900/1799] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:20.885699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:20.895281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:20.952090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab02a9c-1521-46d5-b1ec-d9150f1626fc admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:20.973940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:20.976031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:20.978279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:20.978586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] PATH_INFO: `/v3/users/ce5edb3ad4a24205b3be24914d58aa9a` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:20.983434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5edb3ad4a24205b3be24914d58aa9a)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:20.983434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:21.012895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe943de-6874-47fe-89fd-233c476e2070 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:21.034846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:21.035911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.036553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.037024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.037863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:21.041301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:21.171497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 901/1800] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:20 2022] POST /identity/v3/projects => generated 367 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:21.177537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:21.266167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5edb3ad4a24205b3be24914d58aa9a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.355140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40187da4-76a7-4c03-8918-f33b63b4061d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:21.363924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf4b09-2982-4d35-9cb7-64843fcf96cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:21.364943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4b09-2982-4d35-9cb7-64843fcf96cd admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.366172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4b09-2982-4d35-9cb7-64843fcf96cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.366172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4b09-2982-4d35-9cb7-64843fcf96cd admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.366526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf4b09-2982-4d35-9cb7-64843fcf96cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:21.375547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5edb3ad4a24205b3be24914d58aa9a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.390688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5edb3ad4a24205b3be24914d58aa9a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.390688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:21.390937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf4b09-2982-4d35-9cb7-64843fcf96cd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:21.405898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5edb3ad4a24205b3be24914d58aa9a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.432413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5edb3ad4a24205b3be24914d58aa9a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.448181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 900/1801] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:20 2022] DELETE /identity/v3/users/ce5edb3ad4a24205b3be24914d58aa9a => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:21.448181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:21.464780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:21.550194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 902/1802] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:21 2022] POST /identity/v3/users => generated 290 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:21.552948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22e1d-cc4d-4478-aaf0-65937d64f53a None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.553359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22e1d-cc4d-4478-aaf0-65937d64f53a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.553755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22e1d-cc4d-4478-aaf0-65937d64f53a None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.575903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 903/1803] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 00:35:21 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:35:21.577931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.578280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.579677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.620685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:21.622685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:21.705581 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57315c5-6010-43b1-9d73-560cf4d7271a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:21.716876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:21.718033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.718033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.718227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] PATH_INFO: `/v3/projects/783632082be6442b80cb407e72a36329` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.722122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=783632082be6442b80cb407e72a36329)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:21.727678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:21.777762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 904/1804] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:21.784711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:21.784711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:21.824617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d7c2e-8417-49d0-94b8-599065617ac0 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:21.842659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8b500-72c2-4ad9-a681-627aa18d3963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:21.843398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8b500-72c2-4ad9-a681-627aa18d3963 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.843748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8b500-72c2-4ad9-a681-627aa18d3963 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.844073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8b500-72c2-4ad9-a681-627aa18d3963 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.844821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8b500-72c2-4ad9-a681-627aa18d3963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:21.846522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8b500-72c2-4ad9-a681-627aa18d3963 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:21.868344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 905/1805] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:21.870126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:21.870427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:21.870703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:21.904342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:21.908974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:21.959928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '783632082be6442b80cb407e72a36329', 'request_id': 'req-6dcecc09-4b15-492e-b0e8-199e749c1295'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.967991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '783632082be6442b80cb407e72a36329', 'request_id': 'req-6dcecc09-4b15-492e-b0e8-199e749c1295'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:21.976765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '783632082be6442b80cb407e72a36329', 'request_id': 'req-6dcecc09-4b15-492e-b0e8-199e749c1295'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:22.046783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 906/1806] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:35:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:22.047924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:22.049678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:22.065526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943c2a0-77fc-4dd3-a4db-6a83e5f85d63 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:22.087655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] The token cache is being invalidate because project 783632082be6442b80cb407e72a36329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:22.090894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:22.091325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:22.095215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:22.096622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 901/1807] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:21 2022] DELETE /identity/v3/projects/783632082be6442b80cb407e72a36329 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:22.097514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:22.104881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:22.176386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:22.177022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.177296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.177815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:22.181321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:22.282288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcecc09-4b15-492e-b0e8-199e749c1295 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:22.288206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:22.289309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.289828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.290294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] PATH_INFO: `/v3/projects/1d096c48e7344ba8b5a41aeb19fd12cd/users/5004ff21e6fb4e7f8bfae5891edd8475/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:22.312055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d096c48e7344ba8b5a41aeb19fd12cd, user_id=5004ff21e6fb4e7f8bfae5891edd8475, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:22.315887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:22.376493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:22.381150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:22.398800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 902/1808] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:22 2022] PUT /identity/v3/projects/1d096c48e7344ba8b5a41aeb19fd12cd/users/5004ff21e6fb4e7f8bfae5891edd8475/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:22.403513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:22.405759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:22.415782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 907/1809] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:22.417425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:22.419793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:22.442771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c02b2-465e-42fd-8819-a4441a3059b0 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:22.570825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec236bc-dd95-4cb8-8687-9438d4393264 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:22.592354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:22.595023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.595497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:22.595694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.596277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.596860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.597468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:22.598494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] PATH_INFO: `/v3/users/e9085ce86e354166a00bddf8441c48ba` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:22.599970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:22.605054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9085ce86e354166a00bddf8441c48ba)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:22.614468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:22.616197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:22.621056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:22.636111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 908/1810] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:22.639360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:22.679827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da8ff-c711-4945-9eef-bf077e8ac67d service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:22.715057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:22.718967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.720864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.723842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:22.728462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:22.731621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:22.791739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 909/1811] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:22.794238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:22.836848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed320f-f36f-4a65-b8a0-f1dc9e0613b6 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:22.852425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4158898-6726-40fd-a433-76f17f7fbc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:22.852425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4158898-6726-40fd-a433-76f17f7fbc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.852425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4158898-6726-40fd-a433-76f17f7fbc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.852425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4158898-6726-40fd-a433-76f17f7fbc5d admin admin] PATH_INFO: `/v3/projects/1d096c48e7344ba8b5a41aeb19fd12cd/users/5004ff21e6fb4e7f8bfae5891edd8475/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:22.854863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4158898-6726-40fd-a433-76f17f7fbc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d096c48e7344ba8b5a41aeb19fd12cd, user_id=5004ff21e6fb4e7f8bfae5891edd8475, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:22.860776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4158898-6726-40fd-a433-76f17f7fbc5d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:22.906090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 910/1812] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:22 2022] PUT /identity/v3/projects/1d096c48e7344ba8b5a41aeb19fd12cd/users/5004ff21e6fb4e7f8bfae5891edd8475/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:22.917968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9085ce86e354166a00bddf8441c48ba'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:22.936976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:22.937436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:22.938729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:23.014343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] MFA Rules not processed for user `5004ff21e6fb4e7f8bfae5891edd8475`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:23.015719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:23.026041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9085ce86e354166a00bddf8441c48ba'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:23.029750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9085ce86e354166a00bddf8441c48ba'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:23.053702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:23.075792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9085ce86e354166a00bddf8441c48ba'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:23.122905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9085ce86e354166a00bddf8441c48ba'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:23.159225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 903/1813] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:22 2022] DELETE /identity/v3/users/e9085ce86e354166a00bddf8441c48ba => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:23.163895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 911/1814] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:23.181222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:23.184098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:23.408266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d9102-f9b6-455e-afd7-9b7629c91960 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:23.423011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:23.425728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:23.426947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:23.427755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:23.644573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:23.646383 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:23.660276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 904/1815] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:24.032196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:24.172884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd787bb6-6fae-4ffa-a6a9-2e20456a09e0 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:24.190755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:24.192369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:24.195236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:24.196887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:24.200691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:24.203502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:24.214563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:24.284854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:24.356950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6c5da-d838-4101-bb5f-4c52ddf73f67 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:24.362480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:24.363416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:24.363765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:24.364110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] PATH_INFO: `/v3/projects/8b943671096f4cb28b4d46dd6c8503c9` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:24.365670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b943671096f4cb28b4d46dd6c8503c9)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:24.369573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:24.406240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:24.408113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:24.434720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 912/1816] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:24.558933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b943671096f4cb28b4d46dd6c8503c9', 'request_id': 'req-4c937f99-05ec-428d-a19a-054370919571'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:24.574381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b943671096f4cb28b4d46dd6c8503c9', 'request_id': 'req-4c937f99-05ec-428d-a19a-054370919571'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:24.588140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b943671096f4cb28b4d46dd6c8503c9', 'request_id': 'req-4c937f99-05ec-428d-a19a-054370919571'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:24.678572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] The token cache is being invalidate because project 8b943671096f4cb28b4d46dd6c8503c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:24.683063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:24.683063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:24.683063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 905/1817] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:24 2022] DELETE /identity/v3/projects/8b943671096f4cb28b4d46dd6c8503c9 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:24.688046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:24.704404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de148a6-6a1d-4211-8caa-eee0d23624cd service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:24.836516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:24.841449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:24.841596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:24.842451 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:24.854703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:24.854703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] PATH_INFO: `/v3/users/5af044b562d9437c954cf5e73ad994b3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:24.856484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af044b562d9437c954cf5e73ad994b3)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:24.858681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:24.874074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c937f99-05ec-428d-a19a-054370919571 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:24.890462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:24.891185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:24.891468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:24.891744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] PATH_INFO: `/v3/projects/ff35beac0ae244f99e07a6b2db069e78` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:24.893193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff35beac0ae244f99e07a6b2db069e78)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:24.895307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.038074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:25.096391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff35beac0ae244f99e07a6b2db069e78', 'request_id': 'req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.105444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff35beac0ae244f99e07a6b2db069e78', 'request_id': 'req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.113267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff35beac0ae244f99e07a6b2db069e78', 'request_id': 'req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.116421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af044b562d9437c954cf5e73ad994b3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.142117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af044b562d9437c954cf5e73ad994b3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.151610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af044b562d9437c954cf5e73ad994b3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.187872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] The token cache is being invalidate because project ff35beac0ae244f99e07a6b2db069e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:25.188235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.189044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5fc4872e-aa67-4d9e-92a4-9d95da7b7ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.196695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 906/1818] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:24 2022] DELETE /identity/v3/projects/ff35beac0ae244f99e07a6b2db069e78 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:25.198462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.198716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.198987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.221332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:25.222437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:25.289891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af044b562d9437c954cf5e73ad994b3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.290894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-203e1a6d-08a2-4b58-a314-957407ec4198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af044b562d9437c954cf5e73ad994b3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.298691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 913/1819] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:24 2022] DELETE /identity/v3/users/5af044b562d9437c954cf5e73ad994b3 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:25.318358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 907/1820] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:25.326427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.326729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.327108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.352532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:25.353764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:25.428277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 914/1821] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:25.433339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.443267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7901ba-455b-4669-9ac2-8322bbe9c385 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:25.448473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:25.449073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.449348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.449615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.450256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:25.450552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:25.452164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.464867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 908/1822] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:25.469231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.491270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0f204-d0b8-44dc-9f4e-960a241cc492 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:25.497317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:25.498387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.498665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.498933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.499606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:25.501782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.644164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.644812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 915/1823] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:25 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:25.657806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f136d27-484d-40de-8c1a-06e8a0559811 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:25.736206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:25.736868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.737151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.737413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.738086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:25.740055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.811138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.811138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:25.820728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24f5a4-1bde-4430-8a4d-35368028a9d7 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:25.831783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:25.832830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.833201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.833583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] PATH_INFO: `/v3/projects/2ef13e712abd4e11b61fd59642fe7608` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.834895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef13e712abd4e11b61fd59642fe7608)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:25.837952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.876474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 909/1824] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:25 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:25.880538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.891480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1231-cb1d-40bf-910d-a79cbbb138de admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:25.898955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:25.899839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.900118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.900455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.901070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:25.902696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.914892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 910/1825] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:25.918405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.931189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507d055-cf28-449b-85d0-2105d0396097 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:25.936544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:25.941254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:25.941254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:25.941254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:25.941254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:25.941254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:25.948288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef13e712abd4e11b61fd59642fe7608', 'request_id': 'req-8ef705ed-2815-46b8-8cea-53abc4c288b2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:25.952622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 911/1826] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:25.956670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:25.968612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cf9aa-1943-4d29-8627-68c2d0805f40 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.025497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632f49-7fbd-433b-a846-9962c5da55a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.026168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632f49-7fbd-433b-a846-9962c5da55a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.026442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632f49-7fbd-433b-a846-9962c5da55a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.026778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632f49-7fbd-433b-a846-9962c5da55a0 admin admin] PATH_INFO: `/v3/projects/25453adb42a14a12b4b6600968a69354/users/bb2f8519eb3b4e88b8a6a160949e2635/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.045083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632f49-7fbd-433b-a846-9962c5da55a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25453adb42a14a12b4b6600968a69354, user_id=bb2f8519eb3b4e88b8a6a160949e2635, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.050576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632f49-7fbd-433b-a846-9962c5da55a0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.098900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef13e712abd4e11b61fd59642fe7608', 'request_id': 'req-8ef705ed-2815-46b8-8cea-53abc4c288b2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:26.105607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef13e712abd4e11b61fd59642fe7608', 'request_id': 'req-8ef705ed-2815-46b8-8cea-53abc4c288b2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:26.116913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] The token cache is being invalidate because project 2ef13e712abd4e11b61fd59642fe7608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:26.117303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:26.118036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8ef705ed-2815-46b8-8cea-53abc4c288b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:26.119726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 916/1827] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:25 2022] DELETE /identity/v3/projects/2ef13e712abd4e11b61fd59642fe7608 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:26.124480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb44b0-0fae-4c93-b6db-53960f10bf53 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.124762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb44b0-0fae-4c93-b6db-53960f10bf53 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.125053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb44b0-0fae-4c93-b6db-53960f10bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.144733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-cadb44b0-0fae-4c93-b6db-53960f10bf53 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:26.145921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cadb44b0-0fae-4c93-b6db-53960f10bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:26.154879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 917/1828] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:26.162792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824507-3987-414e-88cd-116202aea3ac None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.163235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824507-3987-414e-88cd-116202aea3ac None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.163985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824507-3987-414e-88cd-116202aea3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.184043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 912/1829] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:25 2022] PUT /identity/v3/projects/25453adb42a14a12b4b6600968a69354/users/bb2f8519eb3b4e88b8a6a160949e2635/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:26.184043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:26.191920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.193003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.193359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6e824507-3987-414e-88cd-116202aea3ac None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:26.194465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.194704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6e824507-3987-414e-88cd-116202aea3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:26.232570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] MFA Rules not processed for user `bb2f8519eb3b4e88b8a6a160949e2635`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:26.233865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:26.267421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 918/1830] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:26.272527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824507-3987-414e-88cd-116202aea3ac None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.282545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824507-3987-414e-88cd-116202aea3ac None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.290108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.290998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.291544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.292675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.293587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:26.293758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.296657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.314036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 919/1831] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:26.315780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 913/1832] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:26.318094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.330233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.330233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:26.335934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2e0a3-1c99-4c38-96db-a66bbe435e6b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.342968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.343706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.344057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.344666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.345547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7a12e-b7ab-4e7f-8340-d4c90997d455 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.345723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.348144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.352455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.352967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.353250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.353548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.354189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.356384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.451096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 920/1833] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:26.473264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.496887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:26.502085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 914/1834] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:26.506725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.544241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97968f6-7066-4af3-96db-3e067665d70f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.572490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b9a79-f40d-48bb-9059-4dd0bc26213b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.577531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.578139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.578465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.578787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.579414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.581408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.604118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.604763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.605046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.605315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.605945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.608542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.671066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 915/1835] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:26.674923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.684954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dd955-1e61-48c1-9850-a50fdbfd275f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.689792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.690390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.690666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.690936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.691568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.693238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.710953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 916/1836] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:26.713856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.723505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590c3b2-5844-4f65-8d51-83668f4a115f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.729051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.729637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.729932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.730190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.730814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.732636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.745618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 921/1837] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/users => generated 294 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:26.745978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 917/1838] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:26.750321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.750450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.761358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a9bc-01bd-4a6a-8925-e930ff12f209 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.762484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a09cb-566e-4d0d-9699-f211e7646160 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.767458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ac709-995c-45b0-874c-fe7c0275b888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.767458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ac709-995c-45b0-874c-fe7c0275b888 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.767458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ac709-995c-45b0-874c-fe7c0275b888 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.771621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ac709-995c-45b0-874c-fe7c0275b888 admin admin] PATH_INFO: `/v3/projects/2e0fe13bc8694782bf49ddac7b6cb92f/users/a65cab7ea0ad4efeb82b357dbdabcfbe/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.774041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.776761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.776995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.779268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.779268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.780792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.789443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ac709-995c-45b0-874c-fe7c0275b888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0fe13bc8694782bf49ddac7b6cb92f, user_id=a65cab7ea0ad4efeb82b357dbdabcfbe, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.792643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ac709-995c-45b0-874c-fe7c0275b888 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.794160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 922/1839] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:26.798516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.809691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca798ea6-cdfc-4557-93cc-2e10c1f25274 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.814967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.815585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.815894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.816162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.816866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.818718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.830902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 923/1840] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:26.835758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:26.847126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3756ec-25aa-4ade-afdb-685254b15e4a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:26.852691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0643f89-9816-4ad8-a634-40c8212fd0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:26.853348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0643f89-9816-4ad8-a634-40c8212fd0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.853626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0643f89-9816-4ad8-a634-40c8212fd0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.853897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0643f89-9816-4ad8-a634-40c8212fd0d9 admin admin] PATH_INFO: `/v3/projects/d5b19116b8fc4f188f29134c0db3f818/users/82a06892015648fc8d35a744e4d8c02f/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.870445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0643f89-9816-4ad8-a634-40c8212fd0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b19116b8fc4f188f29134c0db3f818, user_id=82a06892015648fc8d35a744e4d8c02f, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:26.873549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0643f89-9816-4ad8-a634-40c8212fd0d9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:26.873660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 918/1841] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:26 2022] PUT /identity/v3/projects/2e0fe13bc8694782bf49ddac7b6cb92f/users/a65cab7ea0ad4efeb82b357dbdabcfbe/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:26.886022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.886022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.886022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.915933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] MFA Rules not processed for user `a65cab7ea0ad4efeb82b357dbdabcfbe`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:26.917248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:26.974529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 924/1842] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:26 2022] PUT /identity/v3/projects/d5b19116b8fc4f188f29134c0db3f818/users/82a06892015648fc8d35a744e4d8c02f/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:26.989431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:26.989431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:26.989431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:26.994726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 919/1843] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:26.999574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.021301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] MFA Rules not processed for user `82a06892015648fc8d35a744e4d8c02f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:27.023281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:27.081324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36b24f-f781-4bd7-a219-05aa6f8d8cfd None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.116803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 925/1844] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:27.120511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.151437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.151872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.155843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.155843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.155843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.155843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.209732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a5334-b67f-4cb5-8431-a3d958d847fc None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.215172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.215782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.221325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.221325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.221325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.221325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.238373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:27.267582 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 920/1845] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:26 2022] POST /identity/v3/projects => generated 379 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:27.271975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:27.273858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.313141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 926/1846] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:27 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:27.319247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.455789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403df7c7-de0a-4d68-b648-17078dc4db14 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.513837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac90fb-3678-4cf3-9b1b-f2ba918e40a7 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.521495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.522579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.523634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.524263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.526498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.531075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.557228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.558069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.558346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.558632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.559248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.561328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.641525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 927/1847] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:27 2022] POST /identity/v3/users => generated 300 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:27.645021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.656287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8238989-140b-488a-9f69-9042d0c993fa admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.661354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 921/1848] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:27 2022] POST /identity/v3/users => generated 296 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:27.661488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.661784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.666668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.666668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.666668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.666668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.667679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.676158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 928/1849] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:27.678613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88970d-2f4e-49a0-85d3-818e62a1fe36 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.681176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.684920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.685625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.685969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.686254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.686895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.688542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.693422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aa9f1-4ee8-4eeb-bc18-e3e430bd0451 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.696836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.697491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.697771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.698042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] PATH_INFO: `/v3/projects/f923881c524842529f2d4c0746d48d68/users/fab14824ded24bcca3a5ea51d34250c0/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.700529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 922/1850] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:27.705595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.712014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:27.716116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f923881c524842529f2d4c0746d48d68, user_id=fab14824ded24bcca3a5ea51d34250c0, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.718811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.720246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333cbb8-47cf-45cf-ba96-d8e9663e0aca admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.725183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.725863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.726148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.726421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] PATH_INFO: `/v3/projects/8348cf278cfd40f1bef741be29052026/users/3f9ba67707464244860e4e461a704e0c/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.742305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8348cf278cfd40f1bef741be29052026, user_id=3f9ba67707464244860e4e461a704e0c, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.745456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.819250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 929/1851] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:27 2022] PUT /identity/v3/projects/f923881c524842529f2d4c0746d48d68/users/fab14824ded24bcca3a5ea51d34250c0/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:27.819673 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 923/1852] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:27 2022] PUT /identity/v3/projects/8348cf278cfd40f1bef741be29052026/users/3f9ba67707464244860e4e461a704e0c/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:27.825929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.826506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.890083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be995a-626f-471a-bd45-ee0364238a50 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.894087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23102d31-7b99-4a38-b04d-d17787a2178d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.894991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.895851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.895851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.896145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.898470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.899155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.899870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.900507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.900915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.901087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.901710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.903327 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.912840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 930/1853] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:27.914752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 924/1854] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:27.914980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.918554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:27.926257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949b5e8-4b4a-44c9-a2cf-06705a3079d0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.928194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f9254-77d8-404b-86d2-4cc27197f68a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:27.932148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eef284-8b33-464e-ade4-dc5aa04c90da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.932828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eef284-8b33-464e-ade4-dc5aa04c90da admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.933134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eef284-8b33-464e-ade4-dc5aa04c90da admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.933430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eef284-8b33-464e-ade4-dc5aa04c90da admin admin] PATH_INFO: `/v3/projects/8348cf278cfd40f1bef741be29052026/users/3f9ba67707464244860e4e461a704e0c/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.936526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eef284-8b33-464e-ade4-dc5aa04c90da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8348cf278cfd40f1bef741be29052026, user_id=3f9ba67707464244860e4e461a704e0c, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.937486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad0c9c-f4d6-420e-9765-513f157d4d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:27.938154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad0c9c-f4d6-420e-9765-513f157d4d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.938423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad0c9c-f4d6-420e-9765-513f157d4d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.938556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eef284-8b33-464e-ade4-dc5aa04c90da admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.938697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad0c9c-f4d6-420e-9765-513f157d4d59 admin admin] PATH_INFO: `/v3/projects/f923881c524842529f2d4c0746d48d68/users/fab14824ded24bcca3a5ea51d34250c0/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.940787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad0c9c-f4d6-420e-9765-513f157d4d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f923881c524842529f2d4c0746d48d68, user_id=fab14824ded24bcca3a5ea51d34250c0, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:27.943302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad0c9c-f4d6-420e-9765-513f157d4d59 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:27.949780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 931/1855] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:27 2022] PUT /identity/v3/projects/8348cf278cfd40f1bef741be29052026/users/3f9ba67707464244860e4e461a704e0c/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:27.955397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 925/1856] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:27 2022] PUT /identity/v3/projects/f923881c524842529f2d4c0746d48d68/users/fab14824ded24bcca3a5ea51d34250c0/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:27.965997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:27.968869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.969219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.969610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:27.969764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d189c1f-19e8-4094-9932-45796389179a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:27.970231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d189c1f-19e8-4094-9932-45796389179a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:27.970678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d189c1f-19e8-4094-9932-45796389179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.001006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] MFA Rules not processed for user `fab14824ded24bcca3a5ea51d34250c0`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:28.002131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:28.007161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5d189c1f-19e8-4094-9932-45796389179a None None] MFA Rules not processed for user `3f9ba67707464244860e4e461a704e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:28.008541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5d189c1f-19e8-4094-9932-45796389179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:28.087798 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 926/1857] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:28.096373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 932/1858] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:28.097788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:28.099422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:28.113402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d189c1f-19e8-4094-9932-45796389179a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:28.115914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5d189c1f-19e8-4094-9932-45796389179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:28.145385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.169857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d05a33-e1df-4727-a096-a4daccfbe0f4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:28.173716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:28.174396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:28.174685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:28.174955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.180100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d189c1f-19e8-4094-9932-45796389179a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:28.184167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:28.184735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:28.185013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:28.185124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:28.185282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.186514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:28.197440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:28.201456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:28.211572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 927/1859] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:28.214116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 933/1860] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:28.225885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.284857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:28.342851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:28.356434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b547c-4cd9-4ea3-aff6-5fc5ccbd8233 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:28.362675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65857188-9284-4c8e-b572-4e440bb7abfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:28.363549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857188-9284-4c8e-b572-4e440bb7abfe service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:28.364019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857188-9284-4c8e-b572-4e440bb7abfe service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:28.364353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857188-9284-4c8e-b572-4e440bb7abfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.381153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962a0b0-7773-4562-b6bb-d7916062a522 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:28.389591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.397485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:28.398876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:28.399618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:28.399916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] PATH_INFO: `/v3/users/82a06892015648fc8d35a744e4d8c02f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.401196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a06892015648fc8d35a744e4d8c02f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:28.403983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:28.446505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65857188-9284-4c8e-b572-4e440bb7abfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:28.447978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65857188-9284-4c8e-b572-4e440bb7abfe service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:28.457427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 934/1861] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:28.513269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.587278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.602978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a06892015648fc8d35a744e4d8c02f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:28.670863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a06892015648fc8d35a744e4d8c02f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:28.671555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a06892015648fc8d35a744e4d8c02f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:28.687645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a06892015648fc8d35a744e4d8c02f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:28.698811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a06892015648fc8d35a744e4d8c02f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:28.708003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 928/1862] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:28 2022] DELETE /identity/v3/users/82a06892015648fc8d35a744e4d8c02f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:28.713650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:28.713857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:28.714194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.735709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:28.736982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:28.786864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.801049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 935/1863] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:28.811845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:28.813406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:28.825207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:28.874079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20af3d8-0596-4946-a46e-85c97b3621cf admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:28.877723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:28.878328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:28.878604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:28.878874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:28.888318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:28.891428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:28.899907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 929/1864] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:29.363339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:29.405994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:29.420783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:29.426018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48c8ee-7eb0-4c44-9398-7eece48f1cf6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:29.433660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:29.434319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:29.434600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:29.434919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] PATH_INFO: `/v3/projects/d5b19116b8fc4f188f29134c0db3f818` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:29.436139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b19116b8fc4f188f29134c0db3f818)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:29.438445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:29.603403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b19116b8fc4f188f29134c0db3f818', 'request_id': 'req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:29.704364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b19116b8fc4f188f29134c0db3f818', 'request_id': 'req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:29.711906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b19116b8fc4f188f29134c0db3f818', 'request_id': 'req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:29.723669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] The token cache is being invalidate because project d5b19116b8fc4f188f29134c0db3f818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:29.723957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:29.724631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:29.726499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 936/1865] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:29 2022] DELETE /identity/v3/projects/d5b19116b8fc4f188f29134c0db3f818 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:29.730701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:29.740613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc328b9-e48e-4dc2-b531-ac178f6aa5d8 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:29.800812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:29.801468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:29.801744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:29.802012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] PATH_INFO: `/v3/users/fab14824ded24bcca3a5ea51d34250c0` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:29.803133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fab14824ded24bcca3a5ea51d34250c0)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:29.805088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:30.017274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fab14824ded24bcca3a5ea51d34250c0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.088245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fab14824ded24bcca3a5ea51d34250c0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.088984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fab14824ded24bcca3a5ea51d34250c0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.101957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fab14824ded24bcca3a5ea51d34250c0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.109074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-287af54e-0877-4997-96a5-5116072d2158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fab14824ded24bcca3a5ea51d34250c0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.115251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 930/1866] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:29 2022] DELETE /identity/v3/users/fab14824ded24bcca3a5ea51d34250c0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:30.266063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:30.597422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:30.602121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:30.629373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbba5a-d408-49c4-80dc-b03fc859cd0d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:30.682335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:30.725003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:30.725875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:30.726154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:30.726816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] PATH_INFO: `/v3/projects/f923881c524842529f2d4c0746d48d68` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:30.728847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f923881c524842529f2d4c0746d48d68)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:30.731956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:30.905127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f923881c524842529f2d4c0746d48d68', 'request_id': 'req-a90b543d-2d39-457c-9bed-e97952617560'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.980378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f923881c524842529f2d4c0746d48d68', 'request_id': 'req-a90b543d-2d39-457c-9bed-e97952617560'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:30.999475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f923881c524842529f2d4c0746d48d68', 'request_id': 'req-a90b543d-2d39-457c-9bed-e97952617560'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:31.013125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] The token cache is being invalidate because project f923881c524842529f2d4c0746d48d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:31.013425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:31.014180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a90b543d-2d39-457c-9bed-e97952617560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:31.015832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 937/1867] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:30 2022] DELETE /identity/v3/projects/f923881c524842529f2d4c0746d48d68 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:31.020623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.021102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.021203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.044223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:31.045490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:31.107200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 931/1868] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:31.116270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bf241-def3-4951-811c-7994853a9b55 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.116784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bf241-def3-4951-811c-7994853a9b55 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.117347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bf241-def3-4951-811c-7994853a9b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.139079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-812bf241-def3-4951-811c-7994853a9b55 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:31.140354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-812bf241-def3-4951-811c-7994853a9b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:31.149541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 938/1869] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:31.153826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.163639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec8f9b-833b-4adf-8862-3ec6b73a1b4a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.168338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.168949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.169226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.169502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.170146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:31.170447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:31.172051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:31.194198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 932/1870] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:31.195532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bf241-def3-4951-811c-7994853a9b55 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.205600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bf241-def3-4951-811c-7994853a9b55 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.211096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.211915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.212141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.212642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.213084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:31.215062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:31.306116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:31.313727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 939/1871] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:31 2022] POST /identity/v3/projects => generated 409 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:31.317895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.328579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be156ea2-ed9e-464a-a435-b0b627ce1492 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.399562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.400188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.400523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.400768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.401414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:31.403334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:31.500701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 933/1872] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:31 2022] POST /identity/v3/users => generated 305 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:31.505254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.519709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b174ff-8257-448c-94b0-513ddf797504 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.525480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.526126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.526406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.526711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.527414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:31.529068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:31.546684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 940/1873] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:31.560615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.563471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642631-a66e-4830-bb25-baa8f71ea93e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.578210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a93207-04a7-46f6-bfe9-cb01da22d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.579121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93207-04a7-46f6-bfe9-cb01da22d758 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.579603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93207-04a7-46f6-bfe9-cb01da22d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.580091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93207-04a7-46f6-bfe9-cb01da22d758 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.580995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a93207-04a7-46f6-bfe9-cb01da22d758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:31.582814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a93207-04a7-46f6-bfe9-cb01da22d758 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:31.598867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 934/1874] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:31.604888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.630219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45076-3351-465a-923b-9787ebfccf44 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.636741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.637705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.638190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.638559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] PATH_INFO: `/v3/projects/087655fef1394f289c6af3aaae7f8705/users/041154d3715a4cd6bbc6f150cfeedbaf/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.674968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087655fef1394f289c6af3aaae7f8705, user_id=041154d3715a4cd6bbc6f150cfeedbaf, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:31.677914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:31.746969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 941/1875] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:31 2022] PUT /identity/v3/projects/087655fef1394f289c6af3aaae7f8705/users/041154d3715a4cd6bbc6f150cfeedbaf/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:31.758460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.758460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:31.758873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:31.791259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] MFA Rules not processed for user `041154d3715a4cd6bbc6f150cfeedbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:31.792414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:31.800041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:31.873452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 935/1876] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:31 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:31.878021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:31.982478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:31.992193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006588e-12ff-416f-9b22-fdac9e017ad3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:31.998582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:31.999601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:31.999787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.000071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.000999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.003018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.136848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 942/1877] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:31 2022] POST /identity/v3/projects => generated 409 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:32.141591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:32.155365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5c7a5-7774-4f96-becf-c4544a689dcf None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:32.222228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:32.222228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.223466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.223466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.223466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.225341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.299210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 936/1878] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:32 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:32.300153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:32.311024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd9275-ac9e-423f-b360-0e757ed0cfe3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:32.315993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:32.316775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.317212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.317544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.318265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.320025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.331239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 943/1879] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:32.339930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:32.357366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3ca2f-652d-459d-a1a3-0720ed723010 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:32.362224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:32.362889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.363165 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.363438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] PATH_INFO: `/v3/projects/b0ecfd138cc4491d808d5dc75424591c/users/c6f709f492da4f3eaa8cbec45d59f5b5/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.383586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ecfd138cc4491d808d5dc75424591c, user_id=c6f709f492da4f3eaa8cbec45d59f5b5, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.386319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.494978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 937/1880] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:32 2022] PUT /identity/v3/projects/b0ecfd138cc4491d808d5dc75424591c/users/c6f709f492da4f3eaa8cbec45d59f5b5/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:32.498964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:32.510413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf272359-e427-49fd-9ea7-a3db0abc74c5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:32.576274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a2455-1107-4b02-8aed-c82a5127a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:32.577064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a2455-1107-4b02-8aed-c82a5127a31c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.577363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a2455-1107-4b02-8aed-c82a5127a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.577635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a2455-1107-4b02-8aed-c82a5127a31c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.578279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a2455-1107-4b02-8aed-c82a5127a31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.579857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a2455-1107-4b02-8aed-c82a5127a31c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.591035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 944/1881] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:32.600375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:32.607217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7ea53-a087-4979-a190-a79d503384e9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:32.611981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:32.612669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.612951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.613224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] PATH_INFO: `/v3/projects/b0ecfd138cc4491d808d5dc75424591c/users/c6f709f492da4f3eaa8cbec45d59f5b5/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.617414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ecfd138cc4491d808d5dc75424591c, user_id=c6f709f492da4f3eaa8cbec45d59f5b5, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.620093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.634141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 938/1882] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:32 2022] PUT /identity/v3/projects/b0ecfd138cc4491d808d5dc75424591c/users/c6f709f492da4f3eaa8cbec45d59f5b5/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:32.646227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.646681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.646908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.701665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] MFA Rules not processed for user `c6f709f492da4f3eaa8cbec45d59f5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:32.703464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:32.792693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 945/1883] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:32.806799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:32.808660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:32.881029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42818967-8ce3-4e5e-90ce-a7946f04b973 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:32.885569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:32.886288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:32.886656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:32.886859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:32.897167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:32.898302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:32.913534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 939/1884] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:32 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:33.400203 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:33.402017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:33.418864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe207-6c5c-4d6a-b0a7-984c692aa7e9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:33.423414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:33.424044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:33.424374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:33.424657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:33.484622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:33.501367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:33.503529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:33.515607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 946/1885] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:33 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:34.088406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:34.089956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:34.095361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:34.238227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acfbf6-ddf1-40bb-bcd4-42f48effe336 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:34.248535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:34.250919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:34.250919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:34.251077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] PATH_INFO: `/v3/users/338272c0439044b3936d41d25789314e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:34.252914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338272c0439044b3936d41d25789314e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:34.267064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:34.345118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:34.346822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:34.374938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a82b0-5b49-4af3-b205-7004a3d6cb54 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:34.379967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:34.380654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:34.381138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:34.381455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] PATH_INFO: `/v3/users/bb2f8519eb3b4e88b8a6a160949e2635` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:34.383472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2f8519eb3b4e88b8a6a160949e2635)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:34.385473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:34.579044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338272c0439044b3936d41d25789314e'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.727021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2f8519eb3b4e88b8a6a160949e2635'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.753644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2f8519eb3b4e88b8a6a160949e2635'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.765703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2f8519eb3b4e88b8a6a160949e2635'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.837745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338272c0439044b3936d41d25789314e'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.838473 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338272c0439044b3936d41d25789314e'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.853767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338272c0439044b3936d41d25789314e'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.870183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c32a0e1e-6a01-4202-84b6-ea4f2e4036e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338272c0439044b3936d41d25789314e'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.882246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 940/1886] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:34 2022] DELETE /identity/v3/users/338272c0439044b3936d41d25789314e => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:34.883769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:34.884058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:34.884352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:34.914646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:34.916455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:34.923363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2f8519eb3b4e88b8a6a160949e2635'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.925061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f8edc6d8-b4c8-4350-b7f0-3f928f441695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2f8519eb3b4e88b8a6a160949e2635'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:34.935225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 947/1887] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:34 2022] DELETE /identity/v3/users/bb2f8519eb3b4e88b8a6a160949e2635 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:34.939907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:34.940199 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:34.940508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:34.967940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:34.969746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:34.991977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 941/1888] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:35.000259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:35.002524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:35.046301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:35.054968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 948/1889] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:35.062738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:35.065807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:35.125149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ed4a6-5450-4dff-b4fc-325589ee8ea3 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:35.149803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214ce9d-4e2a-42ea-9035-70ff72b8f2c6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:35.154823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:35.155511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:35.155918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:35.156786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:35.167120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:35.168646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:35.188614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 949/1890] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:35.200064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:35.201329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:35.201792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:35.202564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:35.217802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:35.219847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:35.233883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 942/1891] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:36.112014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:36.113967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:36.128443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:36.134261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd12e2-3b36-4a71-bf59-618ec1a00d18 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:36.140441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:36.141260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:36.141649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:36.142034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] PATH_INFO: `/v3/projects/1360f38a67d442af95a5ce86ef699d6f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:36.143585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1360f38a67d442af95a5ce86ef699d6f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:36.147929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:36.330517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1360f38a67d442af95a5ce86ef699d6f', 'request_id': 'req-cd6c4569-a551-4aed-b52b-a12a707e0b91'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.403126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1360f38a67d442af95a5ce86ef699d6f', 'request_id': 'req-cd6c4569-a551-4aed-b52b-a12a707e0b91'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.410681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1360f38a67d442af95a5ce86ef699d6f', 'request_id': 'req-cd6c4569-a551-4aed-b52b-a12a707e0b91'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.420842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:36.422205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:36.430587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] The token cache is being invalidate because project 1360f38a67d442af95a5ce86ef699d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:36.430931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.431543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.436833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 950/1892] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:36 2022] DELETE /identity/v3/projects/1360f38a67d442af95a5ce86ef699d6f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:36.437773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:36.439172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:36.440011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be1db2-5273-4e34-b2e2-b1cc5dc8a3f9 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:36.508835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:36.509963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:36.510431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:36.510841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] PATH_INFO: `/v3/projects/25453adb42a14a12b4b6600968a69354` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:36.512666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25453adb42a14a12b4b6600968a69354)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:36.515353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:36.534110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c4569-a551-4aed-b52b-a12a707e0b91 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:36.554587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:36.554587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:36.554587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:36.554587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] PATH_INFO: `/v3/users/5004ff21e6fb4e7f8bfae5891edd8475` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:36.554881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5004ff21e6fb4e7f8bfae5891edd8475)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:36.554881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:36.778682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5004ff21e6fb4e7f8bfae5891edd8475'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.785452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25453adb42a14a12b4b6600968a69354', 'request_id': 'req-0d8adbd7-553d-4f0f-aea3-d3123594266a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.794784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5004ff21e6fb4e7f8bfae5891edd8475'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.796361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25453adb42a14a12b4b6600968a69354', 'request_id': 'req-0d8adbd7-553d-4f0f-aea3-d3123594266a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.805930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5004ff21e6fb4e7f8bfae5891edd8475'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.811293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25453adb42a14a12b4b6600968a69354', 'request_id': 'req-0d8adbd7-553d-4f0f-aea3-d3123594266a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.859436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5004ff21e6fb4e7f8bfae5891edd8475'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.860074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-affb92ef-b3f4-45eb-90ba-f0631ca90373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5004ff21e6fb4e7f8bfae5891edd8475'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.867056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 951/1893] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:36 2022] DELETE /identity/v3/users/5004ff21e6fb4e7f8bfae5891edd8475 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:36.868332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebd1d-3745-4d30-b175-8ac80e84cb84 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:36.868640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebd1d-3745-4d30-b175-8ac80e84cb84 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:36.868976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebd1d-3745-4d30-b175-8ac80e84cb84 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:36.877082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 952/1894] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:35:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:35:36.886162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:36.886455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:36.886791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:36.916249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:36.917553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:36.942344 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] The token cache is being invalidate because project 25453adb42a14a12b4b6600968a69354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:36.942900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.945923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:36.947527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 943/1895] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:36 2022] DELETE /identity/v3/projects/25453adb42a14a12b4b6600968a69354 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:36.952892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:36.954979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:36.981869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:37.063956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 953/1896] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:35:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:37.064313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8adbd7-553d-4f0f-aea3-d3123594266a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:37.106908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:37.109000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:37.138384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905ab8a-20f6-4bab-b02a-fa94728c4838 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:37.166366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:37.176257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:37.176257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:37.176257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:37.198559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:37.198559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:37.213250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 954/1897] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:37.244516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:37.245246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:37.245594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:37.245935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] PATH_INFO: `/v3/users/a65cab7ea0ad4efeb82b357dbdabcfbe` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:37.253239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65cab7ea0ad4efeb82b357dbdabcfbe)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:37.255371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:37.471822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65cab7ea0ad4efeb82b357dbdabcfbe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:37.597379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65cab7ea0ad4efeb82b357dbdabcfbe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:37.598007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65cab7ea0ad4efeb82b357dbdabcfbe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:37.606315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:37.607782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:37.624497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65cab7ea0ad4efeb82b357dbdabcfbe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:37.624903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f98d5-6c4d-422b-9ff8-8b543ce99bd4 service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:37.630752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d0741b35-2563-4904-aca4-075808a5698c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65cab7ea0ad4efeb82b357dbdabcfbe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:37.639808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 944/1898] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:36 2022] DELETE /identity/v3/users/a65cab7ea0ad4efeb82b357dbdabcfbe => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:37.649591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:37.650474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:37.650833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:37.651180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] PATH_INFO: `/v3/projects/1d096c48e7344ba8b5a41aeb19fd12cd` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:37.652678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d096c48e7344ba8b5a41aeb19fd12cd)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:37.654753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:37.898645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d096c48e7344ba8b5a41aeb19fd12cd', 'request_id': 'req-069ebef4-84af-4808-a709-68fd7ed54460'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:37.997068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d096c48e7344ba8b5a41aeb19fd12cd', 'request_id': 'req-069ebef4-84af-4808-a709-68fd7ed54460'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.011384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d096c48e7344ba8b5a41aeb19fd12cd', 'request_id': 'req-069ebef4-84af-4808-a709-68fd7ed54460'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.033190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] The token cache is being invalidate because project 1d096c48e7344ba8b5a41aeb19fd12cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:38.033190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.044401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.046366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 955/1899] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:37 2022] DELETE /identity/v3/projects/1d096c48e7344ba8b5a41aeb19fd12cd => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:38.051052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735af3c5-8d73-47fe-b496-156258f1efc6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.051434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735af3c5-8d73-47fe-b496-156258f1efc6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.051801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735af3c5-8d73-47fe-b496-156258f1efc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.082299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-735af3c5-8d73-47fe-b496-156258f1efc6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:38.083122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-735af3c5-8d73-47fe-b496-156258f1efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:38.206862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.208512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:38.228184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 945/1900] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:38.235926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.236306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.236665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.263902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:38.265174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:38.275680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 946/1901] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:38.279820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.290957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208954b-8c82-4ebb-8fe8-597e3418591e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.296732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.297351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.297626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.297897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.298549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:38.299075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.299696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ebef4-84af-4808-a709-68fd7ed54460 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.301112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.306745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.307399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.307682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.307990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] PATH_INFO: `/v3/projects/2e0fe13bc8694782bf49ddac7b6cb92f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.309291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0fe13bc8694782bf49ddac7b6cb92f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.311234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.317145 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 947/1902] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:38.322942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.334767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3637ce7-7a9a-4b4e-b0b0-2dd8bb65cf35 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.340240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.340959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.341229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.341510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.342182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.344109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.442299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0fe13bc8694782bf49ddac7b6cb92f', 'request_id': 'req-4ef85004-1a15-4493-a3fe-14012ddf9c39'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.483222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 948/1903] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:38 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:38.487315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.505806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0fe13bc8694782bf49ddac7b6cb92f', 'request_id': 'req-4ef85004-1a15-4493-a3fe-14012ddf9c39'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.513486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0fe13bc8694782bf49ddac7b6cb92f', 'request_id': 'req-4ef85004-1a15-4493-a3fe-14012ddf9c39'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.526342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] The token cache is being invalidate because project 2e0fe13bc8694782bf49ddac7b6cb92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:38.526646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.527293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.529038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 956/1904] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:38 2022] DELETE /identity/v3/projects/2e0fe13bc8694782bf49ddac7b6cb92f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:38.532906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.538796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:38.538796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:38.560089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cd41a-3971-4c08-b7c4-499b852e73c4 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.565131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.565790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.566072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.566385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.567194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.569714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.612973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85004-1a15-4493-a3fe-14012ddf9c39 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.619140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.619791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.620074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.620365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] PATH_INFO: `/v3/users/3f9ba67707464244860e4e461a704e0c` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.621533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9ba67707464244860e4e461a704e0c)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.623492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.689468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 949/1905] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:38 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:38.695574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.697497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:38.713110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541a841-7dea-4f9f-83bd-dccf85bc4700 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.717973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.718596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.718874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.720303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.720303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.721402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.732174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 950/1906] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:38.735974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.746347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104c38a-38de-4764-8c87-eec67de527e7 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.751352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.751954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.752294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.752602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.753240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.754987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.768545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 951/1907] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:38.770663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:38.781489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684459a1-b282-4130-887c-816a088b26bb admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:38.786184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a834be-7f4a-4d11-9776-27148d9ca009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:38.786809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a834be-7f4a-4d11-9776-27148d9ca009 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.787086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a834be-7f4a-4d11-9776-27148d9ca009 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.787353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a834be-7f4a-4d11-9776-27148d9ca009 admin admin] PATH_INFO: `/v3/projects/374958890f914700848868b5010d4abf/users/4dccd4ccb7b3445ebda4e0bc45a4f87c/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.802694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a834be-7f4a-4d11-9776-27148d9ca009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=374958890f914700848868b5010d4abf, user_id=4dccd4ccb7b3445ebda4e0bc45a4f87c, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:38.805580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a834be-7f4a-4d11-9776-27148d9ca009 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:38.845202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9ba67707464244860e4e461a704e0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.858569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9ba67707464244860e4e461a704e0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.866915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9ba67707464244860e4e461a704e0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.888506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 952/1908] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:38 2022] PUT /identity/v3/projects/374958890f914700848868b5010d4abf/users/4dccd4ccb7b3445ebda4e0bc45a4f87c/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:38.899500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:38.899786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:38.900057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:38.927546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] MFA Rules not processed for user `4dccd4ccb7b3445ebda4e0bc45a4f87c`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:38.929391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:38.967918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9ba67707464244860e4e461a704e0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.969034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9ba67707464244860e4e461a704e0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:38.976630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 957/1909] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:38 2022] DELETE /identity/v3/users/3f9ba67707464244860e4e461a704e0c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:38.998124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 953/1910] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:39.012275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:39.013848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:39.082599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7e866-4548-477c-90aa-d5a34b5807ee admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:39.088809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:39.089242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:39.089525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:39.089802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:39.127906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:39.163666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:39.165296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:39.177994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 958/1911] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:39.201314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:39.203078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:39.287338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad444-94b1-4220-bf6b-adc15a12e560 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:39.354497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:39.355169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:39.355448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:39.355723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] PATH_INFO: `/v3/users/4dccd4ccb7b3445ebda4e0bc45a4f87c` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:39.356953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dccd4ccb7b3445ebda4e0bc45a4f87c)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:39.361290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:39.616791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dccd4ccb7b3445ebda4e0bc45a4f87c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.628082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:39.629763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:39.654946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688a374-eaa3-4d00-a9bf-7b246ae603cc service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:39.660776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:39.665285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:39.665285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:39.665285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] PATH_INFO: `/v3/projects/8348cf278cfd40f1bef741be29052026` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:39.665285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8348cf278cfd40f1bef741be29052026)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:39.667447 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:39.694746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dccd4ccb7b3445ebda4e0bc45a4f87c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.695773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dccd4ccb7b3445ebda4e0bc45a4f87c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.714993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dccd4ccb7b3445ebda4e0bc45a4f87c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.723538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b77acc8-411d-4429-83fb-21083464b890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dccd4ccb7b3445ebda4e0bc45a4f87c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.731103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 954/1912] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:39 2022] DELETE /identity/v3/users/4dccd4ccb7b3445ebda4e0bc45a4f87c => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:39.735301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:39.735594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:39.735872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:39.759884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:39.761173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:39.840457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 955/1913] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:39.870454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8348cf278cfd40f1bef741be29052026', 'request_id': 'req-08337696-1d1a-46ab-b25d-b4a1bfddac2e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.908528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:39.915120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:39.957753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8348cf278cfd40f1bef741be29052026', 'request_id': 'req-08337696-1d1a-46ab-b25d-b4a1bfddac2e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:39.988924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8348cf278cfd40f1bef741be29052026', 'request_id': 'req-08337696-1d1a-46ab-b25d-b4a1bfddac2e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:40.021836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] The token cache is being invalidate because project 8348cf278cfd40f1bef741be29052026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:40.023324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:40.024962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-08337696-1d1a-46ab-b25d-b4a1bfddac2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:40.028383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 959/1914] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:39 2022] DELETE /identity/v3/projects/8348cf278cfd40f1bef741be29052026 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:40.034520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e749f-c432-4b3a-b770-aded80ed52ea None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.034938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e749f-c432-4b3a-b770-aded80ed52ea None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.035411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e749f-c432-4b3a-b770-aded80ed52ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.058209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505858db-4d67-4257-a987-0217f6f3ab9d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.072430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f88e749f-c432-4b3a-b770-aded80ed52ea None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:40.074893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f88e749f-c432-4b3a-b770-aded80ed52ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:40.088352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:40.145497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.146594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.146878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.147820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.150979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:40.176124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 960/1915] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:40.186815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.187263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.187591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.239642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:40.243335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:40.282202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 961/1916] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:40.289100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:40.305985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e654b-bd8f-49f9-a212-7da0964e6412 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.314318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:40.315875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:40.319299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.319928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.320260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.320558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.321219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:40.321516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:40.323271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:40.339301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 956/1917] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:40.348380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 962/1918] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:40.353856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:40.368830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610183a-8f20-4907-a0cc-d27bd283a107 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.384128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.385010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.385612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.386124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.386972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:40.389151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:40.395561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:40.579181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 957/1919] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:40 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:40.583444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:40.594303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f66d8-fc30-409c-8e5f-bd7b8547b7cf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.660847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.661508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.662445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.662445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.663680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:40.665323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:40.815527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 963/1920] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:40 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:40.821756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:40.838744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632b9f6-add8-46a8-a171-34052e1789c7 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.851203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.852934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.853443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.856814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.858436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:40.860396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:40.921002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 958/1921] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:40.924112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:40.942940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeb0ce-380f-438a-bf18-558670744a3f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.949557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.950593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.950955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.951712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:40.952870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:40.954698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:40.968075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 964/1922] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:40.972465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:40.984295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310852e-3dde-4635-9464-1ce2a9db1bca admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:40.989136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01408-349b-40a0-bc1c-a1e9c2e6fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:40.989957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01408-349b-40a0-bc1c-a1e9c2e6fbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:40.990246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01408-349b-40a0-bc1c-a1e9c2e6fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:40.991695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01408-349b-40a0-bc1c-a1e9c2e6fbd1 admin admin] PATH_INFO: `/v3/projects/058dea5289e0472e8bbca762fa65ade3/users/4c9dc645b55c43eeb939f6cd0737f208/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.007474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01408-349b-40a0-bc1c-a1e9c2e6fbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058dea5289e0472e8bbca762fa65ade3, user_id=4c9dc645b55c43eeb939f6cd0737f208, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.010265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01408-349b-40a0-bc1c-a1e9c2e6fbd1 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.094078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.095891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:41.122740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 959/1923] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:40 2022] PUT /identity/v3/projects/058dea5289e0472e8bbca762fa65ade3/users/4c9dc645b55c43eeb939f6cd0737f208/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:41.135244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.136235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.136718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.146667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411222d-57d1-4c79-9772-8bb61404c154 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.179480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] MFA Rules not processed for user `4c9dc645b55c43eeb939f6cd0737f208`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:41.181015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:41.263035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.263811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.264098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.267151 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] PATH_INFO: `/v3/projects/374958890f914700848868b5010d4abf` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.270395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=374958890f914700848868b5010d4abf)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.276754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.294973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 960/1924] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:41.296068 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.322622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac371-986b-4ffc-9fef-1f787548130c None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.336847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.337629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.338223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.338600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.339634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.343271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.466587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374958890f914700848868b5010d4abf', 'request_id': 'req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:41.474373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 961/1925] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:41.480249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.526901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:41.557525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468fb7-40b1-4cab-8793-eebb23637652 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.562871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.563476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.563764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.564016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.564672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.566584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.593897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374958890f914700848868b5010d4abf', 'request_id': 'req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:41.603407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374958890f914700848868b5010d4abf', 'request_id': 'req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:41.620195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] The token cache is being invalidate because project 374958890f914700848868b5010d4abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:41.620195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:41.622514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-43aeb9a7-a9e3-4d0c-a1ca-f2de1c3b9a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:41.623159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 965/1926] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:41 2022] DELETE /identity/v3/projects/374958890f914700848868b5010d4abf => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:41.634114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa09e-0027-4ede-ac40-45653325e48a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.634114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa09e-0027-4ede-ac40-45653325e48a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.634114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa09e-0027-4ede-ac40-45653325e48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.652436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-080aa09e-0027-4ede-ac40-45653325e48a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:41.653768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-080aa09e-0027-4ede-ac40-45653325e48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:41.662484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 966/1927] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:41.671816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.672176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.672527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.677626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 962/1928] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:41.682563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.684477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:41.696470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:41.697956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:41.699048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbe613-4709-404d-8b5a-6b692966b6b3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.704735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.705785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.706322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.706756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.708399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 967/1929] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:41.708516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.709351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.715659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.721777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 963/1930] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:41.726577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af498532-6616-43de-a8a4-6c3fdbda3395 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.727110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.731982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.732819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.733250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.738030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.738030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:41.738030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.738255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:41.738331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.739628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebd5f1-979d-455f-9ec1-102b4a5f98ba admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.744602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.745296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.745571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.745840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] PATH_INFO: `/v3/projects/996efdfe873c4c98b709fdb1c9770c63/users/a322b73ddb0a454887047a9c8001fdce/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.753223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 968/1931] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:41.757810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.768852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc161288-6ed0-4877-b97b-7de3ff58feae admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.775078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] RBAC: Authorizing `identity:create_grant(project_id=996efdfe873c4c98b709fdb1c9770c63, user_id=a322b73ddb0a454887047a9c8001fdce, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.775232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.776041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.776551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.776826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.777456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.778419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.779371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:41.854492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 964/1932] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:41 2022] PUT /identity/v3/projects/996efdfe873c4c98b709fdb1c9770c63/users/a322b73ddb0a454887047a9c8001fdce/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:41.858290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.888375 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 969/1933] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:41.892166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:41.939475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231092c-8ceb-419e-8efc-86d342a0adce admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.974622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09132358-17f6-4a4e-adb2-799343796248 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:41.980193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:41.981137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:41.981515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:41.981890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:41.982775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:41.985878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.045650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.046280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.046558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.046826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.047623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.049317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.061152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 965/1934] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:42.065643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:42.077360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25515af1-25eb-4c1d-baf9-c6523e67b6fe admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:42.084706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a75de-d354-45d9-a254-6c7de14de0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.086399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a75de-d354-45d9-a254-6c7de14de0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.086399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a75de-d354-45d9-a254-6c7de14de0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.086399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a75de-d354-45d9-a254-6c7de14de0c3 admin admin] PATH_INFO: `/v3/projects/996efdfe873c4c98b709fdb1c9770c63/users/a322b73ddb0a454887047a9c8001fdce/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.089847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a75de-d354-45d9-a254-6c7de14de0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996efdfe873c4c98b709fdb1c9770c63, user_id=a322b73ddb0a454887047a9c8001fdce, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.092713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a75de-d354-45d9-a254-6c7de14de0c3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.104840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 966/1935] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:42 2022] PUT /identity/v3/projects/996efdfe873c4c98b709fdb1c9770c63/users/a322b73ddb0a454887047a9c8001fdce/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:42.117322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.117603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.117876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.123302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 970/1936] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:41 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:42.127751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:42.154001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1823-25d4-4ad2-82f0-9669df4e458f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:42.163504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] MFA Rules not processed for user `a322b73ddb0a454887047a9c8001fdce`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:42.165046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:42.231385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.232006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.232277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.232606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.233279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.234878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.245667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 971/1937] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:42.250396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:42.251518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 967/1938] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:42.261580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309891c-ae3a-4c00-bcdc-1f0807959ce9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:42.263319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:42.264718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:42.266422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.267056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.267363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.267637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.268444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.270190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.281649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 972/1939] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:42.285832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:42.298874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6792d-33eb-45ab-a746-bb39a7a2474b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:42.306011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.306679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.306957 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.307317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] PATH_INFO: `/v3/projects/3572fe2ff1724d3fb295a287f2464fbf/users/f6f60f83622c42ae8b523ac7eaaa8f9b/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.323886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3572fe2ff1724d3fb295a287f2464fbf, user_id=f6f60f83622c42ae8b523ac7eaaa8f9b, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.325266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0548c-c6f2-45ef-8c27-585eaf2b1ce8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:42.326591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.329024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf9a81-5eb4-4fe1-b226-acfb26ec4a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.329647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9a81-5eb4-4fe1-b226-acfb26ec4a7a service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.329928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9a81-5eb4-4fe1-b226-acfb26ec4a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.330204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9a81-5eb4-4fe1-b226-acfb26ec4a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.339997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf9a81-5eb4-4fe1-b226-acfb26ec4a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.344132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf9a81-5eb4-4fe1-b226-acfb26ec4a7a service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.352555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 968/1940] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:42.466953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 973/1941] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:42 2022] PUT /identity/v3/projects/3572fe2ff1724d3fb295a287f2464fbf/users/f6f60f83622c42ae8b523ac7eaaa8f9b/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:42.469545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.469831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.470108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.509122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] MFA Rules not processed for user `f6f60f83622c42ae8b523ac7eaaa8f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:42.510229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:42.599522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 969/1942] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:42.612964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:42.614967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:42.742263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30f45e-41fc-412f-8e5f-69c20a9966b5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:42.746928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f29a6-2f71-4ee2-b44c-6abf3fb40f20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:42.747579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f29a6-2f71-4ee2-b44c-6abf3fb40f20 service glance] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:42.747852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f29a6-2f71-4ee2-b44c-6abf3fb40f20 service glance] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:42.748521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f29a6-2f71-4ee2-b44c-6abf3fb40f20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:42.764249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f29a6-2f71-4ee2-b44c-6abf3fb40f20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:42.766879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f29a6-2f71-4ee2-b44c-6abf3fb40f20 service glance] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:42.780550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 974/1943] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:43.237254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:43.486846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:44.020738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:44.418729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:44.494007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e94a2-1803-491f-a9d8-c6e49c561ff8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:44.499840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:44.499840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:44.500995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:44.500995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] PATH_INFO: `/v3/users/4c9dc645b55c43eeb939f6cd0737f208` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:44.501507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9dc645b55c43eeb939f6cd0737f208)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:44.503461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:44.673503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9dc645b55c43eeb939f6cd0737f208'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:44.676571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:44.761843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9dc645b55c43eeb939f6cd0737f208'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:44.762606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9dc645b55c43eeb939f6cd0737f208'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:44.776212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9dc645b55c43eeb939f6cd0737f208'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:44.783169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9dc645b55c43eeb939f6cd0737f208'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:44.790520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 970/1944] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:44 2022] DELETE /identity/v3/users/4c9dc645b55c43eeb939f6cd0737f208 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:44.795006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:44.795205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:44.795506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:44.817209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:44.818475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:44.878527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 975/1945] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:44.888418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:44.889982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:44.901374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:44.951342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab75c15-7f33-4c91-ad34-e8c7beb9e515 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:44.954881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:44.955471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:44.955751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:44.956025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:44.966206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:44.967648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:44.979559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 971/1946] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:45.270825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:45.409742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:45.412265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:45.442405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:45.454870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d6e5e-c64c-44a4-81d8-f3744592552e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:45.461089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:45.461739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:45.462015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:45.462269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] PATH_INFO: `/v3/projects/058dea5289e0472e8bbca762fa65ade3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:45.465141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058dea5289e0472e8bbca762fa65ade3)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:45.467065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:45.585770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058dea5289e0472e8bbca762fa65ade3', 'request_id': 'req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:45.648235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058dea5289e0472e8bbca762fa65ade3', 'request_id': 'req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:45.654844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058dea5289e0472e8bbca762fa65ade3', 'request_id': 'req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:45.667274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] The token cache is being invalidate because project 058dea5289e0472e8bbca762fa65ade3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:45.667558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:45.668240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:45.670090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 976/1947] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:45 2022] DELETE /identity/v3/projects/058dea5289e0472e8bbca762fa65ade3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:45.674074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:45.684542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01632d36-e0eb-43b1-a0b0-899c0a38f71d service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:45.743265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:45.743907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:45.744186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:45.744519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] PATH_INFO: `/v3/users/a322b73ddb0a454887047a9c8001fdce` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:45.745647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a322b73ddb0a454887047a9c8001fdce)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:45.747584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:45.931312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:45.937261 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a322b73ddb0a454887047a9c8001fdce'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:46.008248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a322b73ddb0a454887047a9c8001fdce'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:46.008868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a322b73ddb0a454887047a9c8001fdce'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:46.029962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a322b73ddb0a454887047a9c8001fdce'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:46.037024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e4713f63-f503-48f3-ac78-cdc08981a936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a322b73ddb0a454887047a9c8001fdce'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:46.043503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 972/1948] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:45 2022] DELETE /identity/v3/users/a322b73ddb0a454887047a9c8001fdce => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:46.156781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:47.955113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:47.958348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:47.977012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b966c-b287-42c2-a7f0-6c9a552aa2b4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:48.059351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:48.060009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:48.060296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:48.060613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] PATH_INFO: `/v3/projects/996efdfe873c4c98b709fdb1c9770c63` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:48.061844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996efdfe873c4c98b709fdb1c9770c63)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:48.066595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:48.360971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996efdfe873c4c98b709fdb1c9770c63', 'request_id': 'req-09d214f8-d24a-4a07-a602-2063e1128f19'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:48.458262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996efdfe873c4c98b709fdb1c9770c63', 'request_id': 'req-09d214f8-d24a-4a07-a602-2063e1128f19'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:48.465782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996efdfe873c4c98b709fdb1c9770c63', 'request_id': 'req-09d214f8-d24a-4a07-a602-2063e1128f19'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:48.479146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] The token cache is being invalidate because project 996efdfe873c4c98b709fdb1c9770c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:48.479440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:48.480146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-09d214f8-d24a-4a07-a602-2063e1128f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:48.484996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 977/1949] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:47 2022] DELETE /identity/v3/projects/996efdfe873c4c98b709fdb1c9770c63 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:48.490073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:48.490073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:48.490073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:48.513720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:48.514949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:48.596570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 973/1950] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:48.600384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:48.600871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:48.601198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:48.632358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:48.632358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:48.640537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 978/1951] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:48.644961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:48.658712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7e90-412a-4097-8406-c0f9b83456d2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:48.665097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:48.666060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:48.666500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:48.666934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:48.668050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:48.668589 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:48.671228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:48.688003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 974/1952] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:48.695673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:48.706640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8f3a6-652a-400b-98c9-66f559b70fba None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:48.714185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:48.714792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:48.715139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:48.716594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:48.716594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:48.719386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:48.817267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 979/1953] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:48 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:48.822171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:48.836852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c879707-f0c3-49f4-956c-b9afe924ecd4 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:48.914881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:48.920179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:48.920625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:48.920625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:48.921396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:48.923417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.005483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 975/1954] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:48 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:49.009868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:49.026644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e6044-53f1-4756-ae71-54d6a523340c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:49.032229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:49.032955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.033743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.034207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.035478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:49.038196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.058901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 980/1955] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:49.063600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:49.102089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e1ad8-f196-4907-93e3-dc6ff4bfa0c9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:49.112086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:49.113283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.113783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.114308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.116151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:49.134121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.152831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 976/1956] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:49.156619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:49.179927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174d3e6-c4b1-466a-99c1-94a96add8996 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:49.186690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c363cd0-d3f6-42ce-ac37-2eb38fbb2e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:49.187860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c363cd0-d3f6-42ce-ac37-2eb38fbb2e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.188223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c363cd0-d3f6-42ce-ac37-2eb38fbb2e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.188573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c363cd0-d3f6-42ce-ac37-2eb38fbb2e23 admin admin] PATH_INFO: `/v3/projects/5dde7828737f4f028f7e715b5aed2e29/users/62e743c2f2334203b0bc58e6e27cfb92/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.213520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c363cd0-d3f6-42ce-ac37-2eb38fbb2e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dde7828737f4f028f7e715b5aed2e29, user_id=62e743c2f2334203b0bc58e6e27cfb92, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:49.217674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c363cd0-d3f6-42ce-ac37-2eb38fbb2e23 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.283180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:49.285110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:49.300599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e6ed6-0b10-4ea3-9dbe-0eb03c7acf27 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:49.306321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 981/1957] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:49 2022] PUT /identity/v3/projects/5dde7828737f4f028f7e715b5aed2e29/users/62e743c2f2334203b0bc58e6e27cfb92/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:49.317323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.317742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.318026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.338443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:49.352529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] MFA Rules not processed for user `62e743c2f2334203b0bc58e6e27cfb92`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:49.353784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:49.363379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:49.383899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:49.384695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.385094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.385532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] PATH_INFO: `/v3/users/041154d3715a4cd6bbc6f150cfeedbaf` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.386776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=041154d3715a4cd6bbc6f150cfeedbaf)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:49.388891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.476100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 982/1958] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:49.493882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:49.499071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:49.579394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf376a6-85b2-4530-8eba-3244703e2856 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:49.583620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a2204-b4c5-425d-ad56-16c04218acf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:49.584990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2204-b4c5-425d-ad56-16c04218acf6 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.585269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2204-b4c5-425d-ad56-16c04218acf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.585633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a2204-b4c5-425d-ad56-16c04218acf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.602699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2204-b4c5-425d-ad56-16c04218acf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:49.604208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a2204-b4c5-425d-ad56-16c04218acf6 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.606581 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041154d3715a4cd6bbc6f150cfeedbaf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:49.615354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 983/1959] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:49.694116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041154d3715a4cd6bbc6f150cfeedbaf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:49.695839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041154d3715a4cd6bbc6f150cfeedbaf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:49.712346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041154d3715a4cd6bbc6f150cfeedbaf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:49.719565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041154d3715a4cd6bbc6f150cfeedbaf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:49.728533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 977/1960] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:49 2022] DELETE /identity/v3/users/041154d3715a4cd6bbc6f150cfeedbaf => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:49.733653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.734025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.734407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.759520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:49.760986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:49.845272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 984/1961] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:49.856089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:49.857660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:49.939308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e983-c055-4e27-8a45-f3c2a2dc8abd admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:49.943092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:49.944583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:49.944867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:49.945131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:49.955652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:49.957064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:49.969367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 978/1962] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:51.140732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:51.146100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:51.185910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b22c5-3814-4500-993b-90f817bade20 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:51.188365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:51.189084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:51.189608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:51.190094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] PATH_INFO: `/v3/projects/087655fef1394f289c6af3aaae7f8705` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:51.191756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087655fef1394f289c6af3aaae7f8705)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:51.194306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:51.390771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:51.394007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087655fef1394f289c6af3aaae7f8705', 'request_id': 'req-0fc1c348-bac5-486d-a838-dbaaf94b8362'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:51.504606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087655fef1394f289c6af3aaae7f8705', 'request_id': 'req-0fc1c348-bac5-486d-a838-dbaaf94b8362'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:51.513983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087655fef1394f289c6af3aaae7f8705', 'request_id': 'req-0fc1c348-bac5-486d-a838-dbaaf94b8362'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:51.528922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] The token cache is being invalidate because project 087655fef1394f289c6af3aaae7f8705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:51.529293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:51.529894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:51.532048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 985/1963] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:51 2022] DELETE /identity/v3/projects/087655fef1394f289c6af3aaae7f8705 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:51.536475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:51.561676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90fe2c-fdaf-40b3-8497-dcfc54c2109e service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:51.618963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:51.640561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:51.640748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:51.644801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:51.644801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] PATH_INFO: `/v3/users/c6f709f492da4f3eaa8cbec45d59f5b5` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:51.644801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f709f492da4f3eaa8cbec45d59f5b5)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:51.645714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:51.931352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f709f492da4f3eaa8cbec45d59f5b5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:52.019659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f709f492da4f3eaa8cbec45d59f5b5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:52.019659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f709f492da4f3eaa8cbec45d59f5b5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:52.036252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f709f492da4f3eaa8cbec45d59f5b5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:52.048075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1a148ef8-0ced-4ccf-8880-5831da55f534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f709f492da4f3eaa8cbec45d59f5b5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:52.062167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 979/1964] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:51 2022] DELETE /identity/v3/users/c6f709f492da4f3eaa8cbec45d59f5b5 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:52.779033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:52.781110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:52.814840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1c348-bac5-486d-a838-dbaaf94b8362 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:52.924596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:52.925610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:52.926128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:52.926292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] PATH_INFO: `/v3/projects/b0ecfd138cc4491d808d5dc75424591c` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:52.929188 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ecfd138cc4491d808d5dc75424591c)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:52.931269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.093344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ecfd138cc4491d808d5dc75424591c', 'request_id': 'req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.196608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ecfd138cc4491d808d5dc75424591c', 'request_id': 'req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.205794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ecfd138cc4491d808d5dc75424591c', 'request_id': 'req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.221965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] The token cache is being invalidate because project b0ecfd138cc4491d808d5dc75424591c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:53.222256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.223133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7d8d00ca-a52a-46c9-a4b3-e39ffc720fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.225836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 986/1965] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:52 2022] DELETE /identity/v3/projects/b0ecfd138cc4491d808d5dc75424591c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:53.236009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.236436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.242683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.268778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:53.270012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:53.364484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 980/1966] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:53.367921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.368219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.369184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.390610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:53.391835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:53.401987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 987/1967] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:53.406288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.419214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43df35-e689-4e28-9dc9-41e61a18089e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.424674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.425787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.426050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.426321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.426996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:53.427329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.429254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.462712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 981/1968] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:53.466491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.477516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528be781-cc2d-481a-a9e6-4a842a9b3e84 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.489193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.489832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.490656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.490946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.491804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.493770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.597623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 988/1969] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:53 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:53.602544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.612815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e97fda-c200-4cb8-b552-a015b84a2929 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.668290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:53.677485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.679182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.679675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:53.679790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.680413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.680712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.681351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.686023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.700421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99c7da-7638-41eb-bfb3-50d8a2ef308d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.705751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.706502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.706847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.707189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] PATH_INFO: `/v3/users/7097afa9715849e0b18e46dc8f8b3c79` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.708468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7097afa9715849e0b18e46dc8f8b3c79)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.711332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.782390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 982/1970] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:53 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:53.787041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.798668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6aae-8e3c-412a-9191-a2fb2c4cbd1d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.804675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.805246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.807272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.807272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.807272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.808395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.841860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 983/1971] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:53.841860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.851676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd476c2-874e-45a6-856c-fdfe503f3826 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.857438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.858265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.859869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.859869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.859869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.861435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.881656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 984/1972] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:53.883434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:53.900908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff60a-2fa5-49d7-b8e0-77ec03b041a5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:53.908758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:53.912572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e6001-469e-4191-81b7-1de281c400bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:53.913337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6001-469e-4191-81b7-1de281c400bf admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:53.913698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6001-469e-4191-81b7-1de281c400bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:53.914130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6001-469e-4191-81b7-1de281c400bf admin admin] PATH_INFO: `/v3/projects/f7439fb231524ba78336bc69bed1d957/users/5a54f40416e5472eb481b8252ffb092d/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:53.930964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7097afa9715849e0b18e46dc8f8b3c79'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.931540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e6001-469e-4191-81b7-1de281c400bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7439fb231524ba78336bc69bed1d957, user_id=5a54f40416e5472eb481b8252ffb092d, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:53.935900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e6001-469e-4191-81b7-1de281c400bf admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:53.959213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7097afa9715849e0b18e46dc8f8b3c79'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:53.973174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7097afa9715849e0b18e46dc8f8b3c79'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:54.034716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 985/1973] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:53 2022] PUT /identity/v3/projects/f7439fb231524ba78336bc69bed1d957/users/5a54f40416e5472eb481b8252ffb092d/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:54.038314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.038809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.039287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.076399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7097afa9715849e0b18e46dc8f8b3c79'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:54.078191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c6a5602-0c53-40d1-8c78-2c771357a60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7097afa9715849e0b18e46dc8f8b3c79'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:54.089784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 989/1974] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:53 2022] DELETE /identity/v3/users/7097afa9715849e0b18e46dc8f8b3c79 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:54.091692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.092130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] MFA Rules not processed for user `5a54f40416e5472eb481b8252ffb092d`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:54.092406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.092831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.095837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:54.122793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:54.125626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:54.235418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 986/1975] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:54.236508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.237910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:54.244370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 990/1976] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:54.257320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.257840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee734c-8ba2-4a80-bc40-8f6e16a6f15a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.260863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:54.263552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.264173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.264465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.264905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.265878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.269160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:54.376884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24f54-a327-4209-8d8c-de177a3a838d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.381799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 987/1977] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:54 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:54.386306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.457723 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.459552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.460089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.460645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.486875 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:54.501132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57673-2861-4ea2-aea1-f1f2b8f36343 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.508162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.508928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.509299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.509678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.510415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.512562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:54.524496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:54.551669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.554754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:54.566202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 991/1978] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:54.620767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 988/1979] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:54 2022] POST /identity/v3/users => generated 298 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:54.624855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.634572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90a2e0-1365-4412-905b-4cd8fc47fc8d service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.639254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.639922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.640227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.641816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.642578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.644178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:54.655105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 992/1980] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:54.659595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.670298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccaa72-6e0a-460a-8db3-3910beeeef84 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.675232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.676600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.676879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.677328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] PATH_INFO: `/v3/projects/213da2a924c34140bdc1f44fe55347eb/users/55245e70122249f784c3bc2fcc82aa41/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.694818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213da2a924c34140bdc1f44fe55347eb, user_id=55245e70122249f784c3bc2fcc82aa41, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.697518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:54.823182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 989/1981] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:54 2022] PUT /identity/v3/projects/213da2a924c34140bdc1f44fe55347eb/users/55245e70122249f784c3bc2fcc82aa41/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:54.827860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.841994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad27cdb-cd02-420f-9392-8bf01ab3a619 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.877057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.881032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:54.950257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.951462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.951862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.952204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.952959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.954725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:54.971495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 993/1982] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:54.976189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:54.982969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d98b1-dd6a-4d13-85e5-210462adcc37 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.990099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc38fb-bf5b-4a91-8221-03c230700eb1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:54.990603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:54.991764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:54.992501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:54.992964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] PATH_INFO: `/v3/users/62e743c2f2334203b0bc58e6e27cfb92` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:54.994685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e743c2f2334203b0bc58e6e27cfb92)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:54.998759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:55.003988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481ce5e-02b9-4e4e-a73c-89d5f3e17e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:55.005059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481ce5e-02b9-4e4e-a73c-89d5f3e17e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.005353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481ce5e-02b9-4e4e-a73c-89d5f3e17e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.005966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481ce5e-02b9-4e4e-a73c-89d5f3e17e4f admin admin] PATH_INFO: `/v3/projects/213da2a924c34140bdc1f44fe55347eb/users/55245e70122249f784c3bc2fcc82aa41/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.010370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481ce5e-02b9-4e4e-a73c-89d5f3e17e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=213da2a924c34140bdc1f44fe55347eb, user_id=55245e70122249f784c3bc2fcc82aa41, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:55.014256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481ce5e-02b9-4e4e-a73c-89d5f3e17e4f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:55.036418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 994/1983] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:54 2022] PUT /identity/v3/projects/213da2a924c34140bdc1f44fe55347eb/users/55245e70122249f784c3bc2fcc82aa41/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:55.056634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.056990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.057331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.089318 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] MFA Rules not processed for user `55245e70122249f784c3bc2fcc82aa41`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:55.090521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.173535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 995/1984] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:35:55.181170 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:55.182880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.204719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:55.259938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220081ef-28f1-4979-8375-b0d43feff36b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:55.265701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:55.266441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.266812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.267168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] PATH_INFO: `/v3/projects/632c182513b04837be04a6f87d20ae9d` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.268751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] RBAC: Authorizing `identity:delete_project(project_id=632c182513b04837be04a6f87d20ae9d)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:55.271081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:55.274083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e743c2f2334203b0bc58e6e27cfb92'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.390468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e743c2f2334203b0bc58e6e27cfb92'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.391634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e743c2f2334203b0bc58e6e27cfb92'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.410688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e743c2f2334203b0bc58e6e27cfb92'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.429373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e743c2f2334203b0bc58e6e27cfb92'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.443345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 990/1985] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:54 2022] DELETE /identity/v3/users/62e743c2f2334203b0bc58e6e27cfb92 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:55.448594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:55.448594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.456768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '632c182513b04837be04a6f87d20ae9d', 'request_id': 'req-e636b4c3-f411-4bed-a415-0a978c69033e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.529068 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:55.542973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b5cf3-c5d5-46fd-b374-022a1e49fc99 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:55.566172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '632c182513b04837be04a6f87d20ae9d', 'request_id': 'req-e636b4c3-f411-4bed-a415-0a978c69033e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.568724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:55.571950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.572547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.573048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.574601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '632c182513b04837be04a6f87d20ae9d', 'request_id': 'req-e636b4c3-f411-4bed-a415-0a978c69033e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.575321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.625390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] The token cache is being invalidate because project 632c182513b04837be04a6f87d20ae9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:55.626234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.627215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e636b4c3-f411-4bed-a415-0a978c69033e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:55.633091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 996/1986] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:55 2022] DELETE /identity/v3/projects/632c182513b04837be04a6f87d20ae9d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:55.634573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.634872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.635140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.670296 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:55.671766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.672815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:55.674431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:55.690773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 991/1987] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:55.691807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:55.694599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.750679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 997/1988] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:55.766037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:55.771081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.812118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd56ea6-2aa7-4d56-9d26-6054de5dfd6d service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:55.821501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:55.822139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.822594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.822887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] PATH_INFO: `/v3/users/6091338ea5014c58996a472ed0c7a1e4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.824143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6091338ea5014c58996a472ed0c7a1e4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:55.826183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:55.832553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:55.843772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1b0cd-6d37-4879-9edc-10667e6e5061 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:55.848341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:55.849106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.849897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.850605 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:55.859775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:55.861346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:55.871772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 998/1989] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:35:55.977550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:55.979286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:55.994173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aae2a6-765b-43f5-be6d-8b1f086db756 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:55.998586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:55.999280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:55.999657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:55.999998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:56.001959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:56.077045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6091338ea5014c58996a472ed0c7a1e4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.077388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:56.079382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:56.093014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 999/1990] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:56.167554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6091338ea5014c58996a472ed0c7a1e4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.168194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6091338ea5014c58996a472ed0c7a1e4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.192685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6091338ea5014c58996a472ed0c7a1e4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.203206 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6091338ea5014c58996a472ed0c7a1e4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.212141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 992/1991] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:55 2022] DELETE /identity/v3/users/6091338ea5014c58996a472ed0c7a1e4 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:56.383379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:56.385987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:56.413346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edeb87-8cf2-4117-8891-7da3ab58f792 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:56.517625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:56.518631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:56.524004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:56.524737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] PATH_INFO: `/v3/projects/5dde7828737f4f028f7e715b5aed2e29` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:56.527249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dde7828737f4f028f7e715b5aed2e29)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:56.529590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:56.689100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dde7828737f4f028f7e715b5aed2e29', 'request_id': 'req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.778870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dde7828737f4f028f7e715b5aed2e29', 'request_id': 'req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.781073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:56.782867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:56.785650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dde7828737f4f028f7e715b5aed2e29', 'request_id': 'req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.804317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fb3f9-2343-46a8-9dd3-5bc787fcdfb4 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:56.810535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] The token cache is being invalidate because project 5dde7828737f4f028f7e715b5aed2e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:56.811213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.812352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-c6d92bc3-9274-4276-9e22-4af27dd6d6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:56.814185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1000/1992] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:56 2022] DELETE /identity/v3/projects/5dde7828737f4f028f7e715b5aed2e29 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:56.824727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:56.827937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:56.829698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:56.830087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:56.867665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:56.872534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:56.895159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:56.895849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:56.897029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:56.897029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] PATH_INFO: `/v3/users/5a54f40416e5472eb481b8252ffb092d` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:56.898541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a54f40416e5472eb481b8252ffb092d)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:56.900513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:56.990275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1001/1993] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:56.991899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:56.994420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:57.022898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ff1f-f407-417b-8336-2fdfbdba9fb5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.032390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.033447 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.033726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.034274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] PATH_INFO: `/v3/projects/eef5188ba89c44b0bbfa6b431d34dd28` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.039471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef5188ba89c44b0bbfa6b431d34dd28)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.039471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.043006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:57.139005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a54f40416e5472eb481b8252ffb092d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.182696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef5188ba89c44b0bbfa6b431d34dd28', 'request_id': 'req-94dec561-18ea-46a0-ac9f-b70b31128fae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.247368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a54f40416e5472eb481b8252ffb092d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.248215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a54f40416e5472eb481b8252ffb092d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.267598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a54f40416e5472eb481b8252ffb092d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.278566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dc8c9e05-ab28-4f57-a32f-7828d6eb0648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a54f40416e5472eb481b8252ffb092d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.287962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 993/1994] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:56 2022] DELETE /identity/v3/users/5a54f40416e5472eb481b8252ffb092d => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:57.292082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090f117-b716-4b38-87ce-a9ff521cf2e2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.292082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090f117-b716-4b38-87ce-a9ff521cf2e2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.292082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090f117-b716-4b38-87ce-a9ff521cf2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.311826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0090f117-b716-4b38-87ce-a9ff521cf2e2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:57.313096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0090f117-b716-4b38-87ce-a9ff521cf2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:57.350373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef5188ba89c44b0bbfa6b431d34dd28', 'request_id': 'req-94dec561-18ea-46a0-ac9f-b70b31128fae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.357736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef5188ba89c44b0bbfa6b431d34dd28', 'request_id': 'req-94dec561-18ea-46a0-ac9f-b70b31128fae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.376659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] The token cache is being invalidate because project eef5188ba89c44b0bbfa6b431d34dd28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:57.377051 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.377658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-94dec561-18ea-46a0-ac9f-b70b31128fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:57.379424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1002/1995] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:56 2022] DELETE /identity/v3/projects/eef5188ba89c44b0bbfa6b431d34dd28 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:57.381158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.381403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.381716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.396532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 994/1996] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:57.398558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.398936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.399103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.406071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:57.407301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:57.419609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1003/1997] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:57.419609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.429909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:57.431522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:57.440259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421dcd1-86fa-425e-8c68-6d7172e6128e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.440826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 995/1998] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:57.444977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.445863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:57.452230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.453091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.453091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.453362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.454017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:57.454324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.456099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c2c82-756e-4964-9a5f-5e7d1c556616 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.473834 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1004/1999] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:57.475849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.476187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.476515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.508368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:57.509586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:57.523912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1005/2000] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:57.527929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.538657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e05098-f518-49a2-8d82-cd7ec643e0df None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.544687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.545313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.545594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.545885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.546604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.548679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.552902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e0a27-f8d1-400e-bc58-d5acef7027da None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.565675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.566626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.567207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.567794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.580278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.581792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.593607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 996/2001] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:57.594612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.606455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69278439-8c27-4ff5-bc2f-21915afa0839 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.612626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.613305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.613613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.613881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.614522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:35:57.615021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.616674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.632663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 997/2002] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:35:57.638290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.651961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187226e-d7a3-4eb6-ae94-388543902fbd admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.657122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.658069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.658184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.658414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.659065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.661076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.684701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1006/2003] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:57.686103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.760516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206a324-e0a3-4019-a893-b326e3374e5f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:57.766407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:57.767032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:57.767313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:57.767584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:57.768221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:57.770204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:57.877593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1007/2005] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:57.878799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 998/2005] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:35:57.880075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.881736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:57.903581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d983db-e83f-4f34-b0a5-b36c8341af85 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.009558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7decd97-008b-45ea-934e-8254a37c14e4 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.013503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.014301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.014744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.015420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.016213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.020524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.020854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.021735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.022072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.022401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.024169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.025870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.046224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 999/2006] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:58.050260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.062736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718022b6-8cb1-43c6-9847-151bd3b6f832 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.070135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.070883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.071347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.072129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.072843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.074699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.093126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1000/2007] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:58.095342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.109960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c274cff-c8f2-4acf-9548-1f3354241aae admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.122087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422b65d-5033-4e97-84e9-af50c6f6d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.127967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b65d-5033-4e97-84e9-af50c6f6d8d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.127967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b65d-5033-4e97-84e9-af50c6f6d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.127967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b65d-5033-4e97-84e9-af50c6f6d8d7 admin admin] PATH_INFO: `/v3/projects/2456d44db4d444859f92e9b563c5c9dc/users/a2e39453776b423b91e9c90e02278735/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.128182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1008/2008] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:57 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:58.150530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:58.156438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.157790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422b65d-5033-4e97-84e9-af50c6f6d8d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2456d44db4d444859f92e9b563c5c9dc, user_id=a2e39453776b423b91e9c90e02278735, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.162312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422b65d-5033-4e97-84e9-af50c6f6d8d7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.171818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771408-075d-4593-a225-13d679a4a7a2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.178405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.179185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.179555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.179925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.180998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.182891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.199097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1009/2009] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:58.206472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.240465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02921f13-502c-4a62-910c-cb28ed0c635c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.244016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1001/2010] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:58 2022] PUT /identity/v3/projects/2456d44db4d444859f92e9b563c5c9dc/users/a2e39453776b423b91e9c90e02278735/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:35:58.255375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.256017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.256643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.312781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] MFA Rules not processed for user `a2e39453776b423b91e9c90e02278735`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:58.314790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.330641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:58.378913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.379827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.380377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.380970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.381937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.383889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.397506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1010/2011] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:35:58.399858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.403055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.428070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c979481-226d-4bd7-8115-0c7b50e667ee admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.434288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.435596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.436130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.436611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] PATH_INFO: `/v3/projects/f7439fb231524ba78336bc69bed1d957` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.440087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7439fb231524ba78336bc69bed1d957)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.440639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.443437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1002/2012] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:58.444724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.461256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb5c5b-908a-436e-a0ad-9bd08d20861f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.467949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.468945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.469495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.470022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] PATH_INFO: `/v3/projects/fe2e4e87e5d9414f8a5001d2705a1aa0/users/5280b583fec14939b5a77c7198230e23/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.498508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2e4e87e5d9414f8a5001d2705a1aa0, user_id=5280b583fec14939b5a77c7198230e23, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.501598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.616779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1003/2013] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:58 2022] PUT /identity/v3/projects/fe2e4e87e5d9414f8a5001d2705a1aa0/users/5280b583fec14939b5a77c7198230e23/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:58.618566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.620827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.632551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7439fb231524ba78336bc69bed1d957', 'request_id': 'req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:58.705131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687c4eb-57ba-4923-bbdb-6f05c7f1aaba admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.710115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1601a-f971-45c6-8878-adb761f7206f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.710986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1601a-f971-45c6-8878-adb761f7206f service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.711273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1601a-f971-45c6-8878-adb761f7206f service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.711547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1601a-f971-45c6-8878-adb761f7206f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.727481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7439fb231524ba78336bc69bed1d957', 'request_id': 'req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:58.736317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7439fb231524ba78336bc69bed1d957', 'request_id': 'req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:58.743890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:58.752434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] The token cache is being invalidate because project f7439fb231524ba78336bc69bed1d957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:35:58.752434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:58.753810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6abea0c9-7762-4aaa-a6e7-c2e3269bb4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:58.758303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1011/2014] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:35:58 2022] DELETE /identity/v3/projects/f7439fb231524ba78336bc69bed1d957 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:58.758303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb93dd-cff6-4ee7-9373-05a0625afe55 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.758303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb93dd-cff6-4ee7-9373-05a0625afe55 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.758303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb93dd-cff6-4ee7-9373-05a0625afe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.789077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1601a-f971-45c6-8878-adb761f7206f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.790721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1601a-f971-45c6-8878-adb761f7206f service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.795478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c6eb93dd-cff6-4ee7-9373-05a0625afe55 None None] MFA Rules not processed for user `5280b583fec14939b5a77c7198230e23`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:58.797692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c6eb93dd-cff6-4ee7-9373-05a0625afe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.805189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1004/2015] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:58.806642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b334d81-9d87-4ab0-87d4-103b2fd03aa7 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.806933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b334d81-9d87-4ab0-87d4-103b2fd03aa7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.807202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b334d81-9d87-4ab0-87d4-103b2fd03aa7 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.812424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:INSERT=6,keystone:DELETE=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:58.812558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1005/2016] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:35:58.813816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b334d81-9d87-4ab0-87d4-103b2fd03aa7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.818650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4b334d81-9d87-4ab0-87d4-103b2fd03aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.894732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1012/2017] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:58.896867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.897371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.898035 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.925724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:35:58.927859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.931371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b334d81-9d87-4ab0-87d4-103b2fd03aa7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.936703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.939788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.939788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.939788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] PATH_INFO: `/v3/users/55245e70122249f784c3bc2fcc82aa41` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.939788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55245e70122249f784c3bc2fcc82aa41)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.941566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.973989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:59.020464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1013/2018] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:35:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:35:59.022052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:59.023759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:59.043285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191605-0e4d-439a-8175-e993125a6f90 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:59.054042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:59.054798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:59.055221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:59.055396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:59.056075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:59.058298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:59.170902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55245e70122249f784c3bc2fcc82aa41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:59.197396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1014/2019] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:35:59 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:59.199024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:59.201155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:59.252383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:59.292689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55245e70122249f784c3bc2fcc82aa41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:59.294475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1c600-400b-4f07-897f-fec98b45ec15 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:59.301262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55245e70122249f784c3bc2fcc82aa41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:59.324881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55245e70122249f784c3bc2fcc82aa41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:59.341407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:59.342043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:59.342419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:59.342695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:59.359753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55245e70122249f784c3bc2fcc82aa41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:35:59.376749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:59.376962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1006/2020] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:35:58 2022] DELETE /identity/v3/users/55245e70122249f784c3bc2fcc82aa41 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:35:59.377867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:59.378703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:59.383218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:59.414754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1015/2021] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:59.508809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669113ea-89cf-4a1b-b12b-b296306d7473 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:59.517415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:59.520191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:59.520191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:59.520191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:59.520191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:59.523092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:59.709263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1007/2022] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:35:59 2022] POST /identity/v3/users => generated 295 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:35:59.710652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:59.814943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9ce9-be5c-4d02-b150-bfcfbb4bf7ec service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:59.823345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:59.824841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:59.825232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:59.825601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:59.826353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:59.829201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:59.863869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:59.865542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1016/2023] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:35:59.877381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0c3a4-744b-406d-9b3d-f68e9ad3825c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:59.890407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:59.892395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:59.892889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:59.893377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] PATH_INFO: `/v3/projects/2098701f7e9245909ebe2b53e057a8fb/users/afa7fe025e494bd4a46c236debc89d75/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:59.911902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2098701f7e9245909ebe2b53e057a8fb, user_id=afa7fe025e494bd4a46c236debc89d75, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:59.915347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:00.036639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1008/2024] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:35:59 2022] PUT /identity/v3/projects/2098701f7e9245909ebe2b53e057a8fb/users/afa7fe025e494bd4a46c236debc89d75/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:00.040811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:00.055110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5041394-0618-491e-886f-634b0e151742 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:00.138618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:00.139246 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.139581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.139863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.140548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:00.142256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:00.154206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1017/2025] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:36:00.158057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:00.170266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e91d-cdbb-4085-b576-e6514bdb827f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:00.175341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b9780-73f6-4f28-afeb-be9a0649c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:00.175984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9780-73f6-4f28-afeb-be9a0649c409 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.176264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9780-73f6-4f28-afeb-be9a0649c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.176644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b9780-73f6-4f28-afeb-be9a0649c409 admin admin] PATH_INFO: `/v3/projects/2098701f7e9245909ebe2b53e057a8fb/users/afa7fe025e494bd4a46c236debc89d75/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.178899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b9780-73f6-4f28-afeb-be9a0649c409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2098701f7e9245909ebe2b53e057a8fb, user_id=afa7fe025e494bd4a46c236debc89d75, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:00.181610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b9780-73f6-4f28-afeb-be9a0649c409 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:00.193853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:00.194989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1009/2026] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:00 2022] PUT /identity/v3/projects/2098701f7e9245909ebe2b53e057a8fb/users/afa7fe025e494bd4a46c236debc89d75/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:00.199462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:00.206504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.206784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.207058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.225761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5e23b-d94e-4fb6-bee7-8a7dd1508dcf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:00.244026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] MFA Rules not processed for user `afa7fe025e494bd4a46c236debc89d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:00.245221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:00.297504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=8,keystone:SELECT=86,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:00.304920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:00.313153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:00.313807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.314145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.314809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] PATH_INFO: `/v3/projects/213da2a924c34140bdc1f44fe55347eb` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.315732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213da2a924c34140bdc1f44fe55347eb)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:00.318993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:00.343657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1010/2027] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:00.351746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:00.353491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:00.422697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eae554-b159-4c52-965c-836d8a0beb3a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:00.426396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:00.428963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.428963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.428963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.434092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:00.492637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213da2a924c34140bdc1f44fe55347eb', 'request_id': 'req-26720cc2-40c4-46b7-b5c6-73463f87edb3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:00.494862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:00.496661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdafdf-a3e2-4b48-ab2e-1aaa0b9a2940 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:00.507091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1011/2028] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:00.585179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213da2a924c34140bdc1f44fe55347eb', 'request_id': 'req-26720cc2-40c4-46b7-b5c6-73463f87edb3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:00.593186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213da2a924c34140bdc1f44fe55347eb', 'request_id': 'req-26720cc2-40c4-46b7-b5c6-73463f87edb3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:00.609348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] The token cache is being invalidate because project 213da2a924c34140bdc1f44fe55347eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:00.610514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:00.612949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-26720cc2-40c4-46b7-b5c6-73463f87edb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:00.615047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1018/2029] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:00 2022] DELETE /identity/v3/projects/213da2a924c34140bdc1f44fe55347eb => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:00.620172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.620487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.620786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.645016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:00.645016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:00.709357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1012/2030] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:00.717532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681efc-fffb-4308-9452-a589205efba4 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.717816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681efc-fffb-4308-9452-a589205efba4 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.718116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681efc-fffb-4308-9452-a589205efba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.739213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-01681efc-fffb-4308-9452-a589205efba4 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:00.741346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-01681efc-fffb-4308-9452-a589205efba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:00.756892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1019/2031] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:00.758006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:00.778774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d58232-e3fa-4d83-967e-84b3f33fc171 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:00.778774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:00.778774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.778774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.779187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.779187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:00.779187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:00.779187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:00.791113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1013/2032] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:36:00.795698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681efc-fffb-4308-9452-a589205efba4 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:00.806654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681efc-fffb-4308-9452-a589205efba4 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:00.811665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:00.812505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:00.812833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:00.813243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:00.813912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:00.815849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:01.001786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1020/2033] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:00 2022] POST /identity/v3/projects => generated 371 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:01.002188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:01.024390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b279-3684-4049-91c4-af6d38e88738 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:01.053409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:01.114038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:01.115770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.116133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.116683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.118041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:01.120870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:01.284706 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1014/2034] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:00 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:01.285920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:01.298092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cf8c3-3096-4fd7-aab7-2b8e85fa6a3c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:01.303609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:01.304343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.304700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.305048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.305760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:01.307438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:01.318076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1021/2035] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:36:01.323247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:01.335044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0465-5eb4-40a6-b6bf-c763c4426ec6 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:01.340036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d631e5-b57c-4e14-9cb7-139dc32e9e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:01.340682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d631e5-b57c-4e14-9cb7-139dc32e9e7f admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.340961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d631e5-b57c-4e14-9cb7-139dc32e9e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.341233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d631e5-b57c-4e14-9cb7-139dc32e9e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.341877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d631e5-b57c-4e14-9cb7-139dc32e9e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:01.343561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d631e5-b57c-4e14-9cb7-139dc32e9e7f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:01.355348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1015/2036] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:01.359522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:01.370832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de8132-be51-40ed-99e1-84e51c1e6266 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:01.375621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:01.376408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.377401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.377401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] PATH_INFO: `/v3/projects/bec7340f5f78461a91a3a16390314b2e/users/a4f8919e382f414a8be185ac47d28372/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.406053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec7340f5f78461a91a3a16390314b2e, user_id=a4f8919e382f414a8be185ac47d28372, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:01.411294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:01.551128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1022/2037] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:01 2022] PUT /identity/v3/projects/bec7340f5f78461a91a3a16390314b2e/users/a4f8919e382f414a8be185ac47d28372/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:01.561811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.562101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.562374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.593006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] MFA Rules not processed for user `a4f8919e382f414a8be185ac47d28372`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:01.594293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:01.670607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1016/2038] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:01.685900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:01.685900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:01.763534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d7fa-fecb-47d9-b2af-c2ec50d73369 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:01.769707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d6668-3da0-4334-89ae-68c4863ddd99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:01.770361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d6668-3da0-4334-89ae-68c4863ddd99 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.770808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d6668-3da0-4334-89ae-68c4863ddd99 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.772471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d6668-3da0-4334-89ae-68c4863ddd99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.787902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d6668-3da0-4334-89ae-68c4863ddd99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:01.790425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d6668-3da0-4334-89ae-68c4863ddd99 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:01.804487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1023/2039] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:01.832524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:01.918514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a27ee3-7b8a-4de7-affe-8a6c3d0f7081 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:01.923479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:01.924215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:01.924621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:01.924907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] PATH_INFO: `/v3/users/a4f8919e382f414a8be185ac47d28372` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:01.926208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f8919e382f414a8be185ac47d28372)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:01.928348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:02.075977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:02.232022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f8919e382f414a8be185ac47d28372'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:02.351704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f8919e382f414a8be185ac47d28372'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:02.352405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f8919e382f414a8be185ac47d28372'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:02.365779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f8919e382f414a8be185ac47d28372'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:02.373522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f8919e382f414a8be185ac47d28372'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:02.380820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1017/2040] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:01 2022] DELETE /identity/v3/users/a4f8919e382f414a8be185ac47d28372 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:02.385601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:02.386616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:02.387038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:02.422548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:02.424448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:02.463485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:02.549312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1024/2041] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:02.567254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:02.572558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:02.666356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27988ed-d519-44b1-a0d5-c4237dacd8ad admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:02.670416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271123d-7739-4ce3-98e7-ff9c815c6618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:02.671625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271123d-7739-4ce3-98e7-ff9c815c6618 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:02.671901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271123d-7739-4ce3-98e7-ff9c815c6618 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:02.672171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271123d-7739-4ce3-98e7-ff9c815c6618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:02.685185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271123d-7739-4ce3-98e7-ff9c815c6618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:02.686654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271123d-7739-4ce3-98e7-ff9c815c6618 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:02.698678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1018/2042] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:03.190428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:03.191866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:03.203994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a6071-933c-4129-aa70-4d046c9b275e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:03.210429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:03.211069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:03.211351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:03.211622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] PATH_INFO: `/v3/projects/bec7340f5f78461a91a3a16390314b2e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:03.212965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec7340f5f78461a91a3a16390314b2e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:03.214882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:03.384764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec7340f5f78461a91a3a16390314b2e', 'request_id': 'req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:03.498544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec7340f5f78461a91a3a16390314b2e', 'request_id': 'req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:03.507486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec7340f5f78461a91a3a16390314b2e', 'request_id': 'req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:03.519901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] The token cache is being invalidate because project bec7340f5f78461a91a3a16390314b2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:03.520199 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:03.524879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e6ef7a52-2bf5-4ed5-991a-70c1b70350f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:03.524879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1025/2043] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:03 2022] DELETE /identity/v3/projects/bec7340f5f78461a91a3a16390314b2e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:03.528541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:03.528967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:03.529146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:03.549771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:03.550947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:03.616545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1019/2044] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:03.625356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:03.625667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:03.625944 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:03.650862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:03.652088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:03.661098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1026/2045] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:03.665521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:03.675952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d5aa-bae8-4c83-94d5-74b8db8f7ac3 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:03.685216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:03.685216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:03.685216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:03.685216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:03.685642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:03.685642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:03.685642 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:03.707959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1020/2046] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:03.709678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:03.720011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d22ca8-968c-455a-a4ff-c792b8fef6e3 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:03.725242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:03.725836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:03.726118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:03.726392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:03.727118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:03.731103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:03.842804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1027/2047] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:03 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:03.847175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:03.858472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338d30-2e39-4ce6-b9a8-84945588c968 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:03.891471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:03.917779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:03.918388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:03.918663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:03.918934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:03.919564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:03.921709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:04.005793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1021/2048] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:03 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:04.010660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:04.024272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abfddc-0346-4d4e-ac31-2a31b79fe8cb admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:04.030882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:04.031967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:04.032392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:04.032662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:04.033438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:04.035258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:04.049727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1028/2049] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:04.050946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:04.062873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a854-a786-4ed6-8f7c-9d26d48a6cdc admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:04.068561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163479b1-a81a-4efe-a3f8-913264825bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:04.069472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163479b1-a81a-4efe-a3f8-913264825bfa admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:04.069755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163479b1-a81a-4efe-a3f8-913264825bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:04.070029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163479b1-a81a-4efe-a3f8-913264825bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:04.070678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163479b1-a81a-4efe-a3f8-913264825bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:04.072282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163479b1-a81a-4efe-a3f8-913264825bfa admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:04.084268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1022/2050] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:04.088654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:04.099632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdb282-c4b6-4e4a-98a9-8ccceac946e1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:04.105024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:04.105703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:04.105983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:04.106260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] PATH_INFO: `/v3/projects/ebe5283d88b44f8b824c7bbca56b3473/users/022c4688d9ce4df9a99ae5fa84e1d871/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:04.129434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5283d88b44f8b824c7bbca56b3473, user_id=022c4688d9ce4df9a99ae5fa84e1d871, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:04.133337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:04.201907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1029/2051] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:04 2022] PUT /identity/v3/projects/ebe5283d88b44f8b824c7bbca56b3473/users/022c4688d9ce4df9a99ae5fa84e1d871/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:04.206788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:04.207091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:04.207438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:04.227791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:04.229136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:04.291631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1023/2052] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:04.301839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:04.304479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:04.355634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:04.367419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ded48c-844b-403c-bb36-61ee0553b1cf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:04.371338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb70ef-9576-4391-9323-8f9cdc042305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:04.371995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb70ef-9576-4391-9323-8f9cdc042305 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:04.372369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb70ef-9576-4391-9323-8f9cdc042305 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:04.372719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb70ef-9576-4391-9323-8f9cdc042305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:04.383085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb70ef-9576-4391-9323-8f9cdc042305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:04.384560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb70ef-9576-4391-9323-8f9cdc042305 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:04.394745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1030/2053] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:08.316067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:08.318366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:08.403734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542194d-ae7a-4d2c-8e8e-e00852a3c3aa None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:08.407679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:08.408351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:08.408850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:08.409127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:08.410933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:08.502817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:08.504470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:08.517513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1024/2054] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:36:10.638836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:10.639343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:10.639419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:10.696419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] MFA Rules not processed for user `022c4688d9ce4df9a99ae5fa84e1d871`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:10.698560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:10.837565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1031/2055] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:10.842180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:10.849142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:10.857402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482f6fc-4d0a-409e-8f18-28c094ac3f56 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:10.866324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:10.867130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:10.867435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:10.867908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:10.868647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:10.870597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:10.983134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1025/2056] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:10 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:10.987438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:11.004028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107093d7-cc50-4f07-8dbb-c0d8c372e645 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:11.084644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:11.090418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:11.090418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:11.090418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:11.090418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:11.090418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:11.200392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1032/2057] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:10 2022] POST /identity/v3/users => generated 295 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:11.200610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:11.212179 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c10c9-b7e1-4fd5-b273-420c04849610 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:11.225471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:11.231204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:11.231204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:11.231204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:11.231960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:11.234549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:11.258898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1026/2058] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:36:11.260401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:11.272311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fc377-fe9f-4254-8ef9-0da4be782e9a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:11.277733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:11.278693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:11.279139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:11.280067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] PATH_INFO: `/v3/projects/79152a9312394a609050dbeb63c36310/users/bc49646f0846434f9fa3931b5db0f517/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:11.298492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79152a9312394a609050dbeb63c36310, user_id=bc49646f0846434f9fa3931b5db0f517, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:11.301523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:11.385814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1033/2059] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:11 2022] PUT /identity/v3/projects/79152a9312394a609050dbeb63c36310/users/bc49646f0846434f9fa3931b5db0f517/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:11.389916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:11.401157 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f41103-b4a4-44ab-ac8c-c64ae40a3952 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:11.470733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:11.471464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:11.471743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:11.472013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:11.472710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:11.474984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:11.489293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1027/2060] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:11.490135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:11.503271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df9db5-f313-41b8-ab97-799e9507e8e0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:11.513748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fb06d-b9f5-476b-9c98-37debdcb01c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:11.514767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fb06d-b9f5-476b-9c98-37debdcb01c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:11.515061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fb06d-b9f5-476b-9c98-37debdcb01c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:11.515368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fb06d-b9f5-476b-9c98-37debdcb01c4 admin admin] PATH_INFO: `/v3/projects/79152a9312394a609050dbeb63c36310/users/bc49646f0846434f9fa3931b5db0f517/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:11.517453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fb06d-b9f5-476b-9c98-37debdcb01c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79152a9312394a609050dbeb63c36310, user_id=bc49646f0846434f9fa3931b5db0f517, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:11.520841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fb06d-b9f5-476b-9c98-37debdcb01c4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:11.533518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1034/2061] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:11 2022] PUT /identity/v3/projects/79152a9312394a609050dbeb63c36310/users/bc49646f0846434f9fa3931b5db0f517/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:14.559652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:14.562200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:14.579738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f70f6-3df7-4aba-bd05-1790c0457cd5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:14.690156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:14.693091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:14.694237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:14.694732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] PATH_INFO: `/v3/users/f6f60f83622c42ae8b523ac7eaaa8f9b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:14.696948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6f60f83622c42ae8b523ac7eaaa8f9b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:14.699187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:14.891024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6f60f83622c42ae8b523ac7eaaa8f9b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:14.977956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6f60f83622c42ae8b523ac7eaaa8f9b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:14.978725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6f60f83622c42ae8b523ac7eaaa8f9b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:14.997924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:15.001880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6f60f83622c42ae8b523ac7eaaa8f9b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:15.030790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6f60f83622c42ae8b523ac7eaaa8f9b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:15.044845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1028/2062] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:14 2022] DELETE /identity/v3/users/f6f60f83622c42ae8b523ac7eaaa8f9b => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:15.049350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:15.049899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:15.050052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:15.075609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:15.077226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:15.115473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:15.189259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1035/2063] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:15.201690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:15.203256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:15.353159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c0beb-2731-49dc-ac9b-bb5771447849 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:15.362527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2264591-cc18-4294-b42e-80c0da19d28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:15.363212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264591-cc18-4294-b42e-80c0da19d28f service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:15.363501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264591-cc18-4294-b42e-80c0da19d28f service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:15.365700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2264591-cc18-4294-b42e-80c0da19d28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:15.395445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2264591-cc18-4294-b42e-80c0da19d28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:15.397211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2264591-cc18-4294-b42e-80c0da19d28f service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:15.435064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1029/2064] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:16.252016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:16.255588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:16.272316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477bd5e-ab39-405a-a94b-fc9dc3531cf2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:16.278572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:16.279547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:16.280068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:16.282498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] PATH_INFO: `/v3/projects/3572fe2ff1724d3fb295a287f2464fbf` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:16.284245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3572fe2ff1724d3fb295a287f2464fbf)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:16.288490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:16.432316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3572fe2ff1724d3fb295a287f2464fbf', 'request_id': 'req-3049d596-d79f-4a38-9b51-e07aefd35682'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:16.584297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3572fe2ff1724d3fb295a287f2464fbf', 'request_id': 'req-3049d596-d79f-4a38-9b51-e07aefd35682'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:16.592720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3572fe2ff1724d3fb295a287f2464fbf', 'request_id': 'req-3049d596-d79f-4a38-9b51-e07aefd35682'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:16.607947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] The token cache is being invalidate because project 3572fe2ff1724d3fb295a287f2464fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:16.608520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:16.610223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3049d596-d79f-4a38-9b51-e07aefd35682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:16.612248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1036/2065] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:16 2022] DELETE /identity/v3/projects/3572fe2ff1724d3fb295a287f2464fbf => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:16.617580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:16.618276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:16.618276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:16.648239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:16.649799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:16.741988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1030/2066] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:16.750490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:16.755896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:16.755896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:16.783418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:16.784769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:16.796100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1037/2067] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:16.799877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:16.814741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814c6a-3dce-4032-a57b-79f7362d60c7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:16.821868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:16.822902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:16.823373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:16.831008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:16.831008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:16.831008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:16.831008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:16.849075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:16.852552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1031/2068] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:16.859211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a62f1-0048-40f7-8034-6c46111b640a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:16.866274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:16.866899 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:16.867177 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:16.867480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:16.868130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:16.870077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:16.999335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1038/2069] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:16 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:17.003903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:17.023045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736edf7d-d816-462a-969d-e9dc897f2fe1 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:17.095223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:17.095931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:17.096222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:17.096525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:17.097180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:17.099452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:17.195209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1032/2070] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:17 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:17.199492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:17.219239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd6ec4-c857-4652-922e-8cf8345df66e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:17.239655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:17.240824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:17.241287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:17.241732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:17.244098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:17.247141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:17.264652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1039/2071] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:17.269345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:17.278349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:17.282201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a3061-c140-4397-8e38-dbc37bbfc344 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:17.287786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a2d4c-ff3b-464b-b81b-9424d7345476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:17.288281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2d4c-ff3b-464b-b81b-9424d7345476 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:17.288620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2d4c-ff3b-464b-b81b-9424d7345476 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:17.288900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2d4c-ff3b-464b-b81b-9424d7345476 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:17.289544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a2d4c-ff3b-464b-b81b-9424d7345476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:17.291382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a2d4c-ff3b-464b-b81b-9424d7345476 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:17.309846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1033/2072] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:17.313935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:17.325815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e098-c0fe-4534-8885-6325dbd4c10e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:17.331040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:17.331713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:17.331994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:17.332271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] PATH_INFO: `/v3/projects/2d1582f5fce54f96b6c641f79b6d5833/users/05bc430fed28409b8f3aee71c0e6cc1b/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:17.351373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1582f5fce54f96b6c641f79b6d5833, user_id=05bc430fed28409b8f3aee71c0e6cc1b, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:17.354229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:17.461901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1040/2073] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:17 2022] PUT /identity/v3/projects/2d1582f5fce54f96b6c641f79b6d5833/users/05bc430fed28409b8f3aee71c0e6cc1b/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:17.472771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c178b-c32a-4961-befc-39fd058f8a0e None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:17.473033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c178b-c32a-4961-befc-39fd058f8a0e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:17.474295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c178b-c32a-4961-befc-39fd058f8a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:17.517771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-403c178b-c32a-4961-befc-39fd058f8a0e None None] MFA Rules not processed for user `05bc430fed28409b8f3aee71c0e6cc1b`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:17.519205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-403c178b-c32a-4961-befc-39fd058f8a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:17.635916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1034/2074] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:17.647922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:17.651511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:17.719661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:17.724925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1e47-62d8-4642-a1b1-a215fdbb5284 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:17.728821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:17.729433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:17.729712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:17.729986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:17.739760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:17.741228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:17.751861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1041/2075] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:18.219550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:18.219830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:18.220147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:18.252818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] MFA Rules not processed for user `bc49646f0846434f9fa3931b5db0f517`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:18.253978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:18.365851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1035/2076] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:18.380471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:18.382253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:18.522078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ff66d-ac6f-4740-a4c5-439561e03d4e service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:18.531462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:18.532388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:18.532850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:18.533275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:18.535421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:18.611886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:18.613782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a266e-4348-454b-bb3b-7fb14d72e11c service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:18.624615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1042/2077] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:20.974040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:20.975679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:21.074739 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9809196-465a-4c08-96d7-a07cf469e6e1 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:21.080279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:21.080967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:21.081246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:21.081517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] PATH_INFO: `/v3/users/a2e39453776b423b91e9c90e02278735` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:21.082749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e39453776b423b91e9c90e02278735)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:21.084875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:21.097681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:21.320028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e39453776b423b91e9c90e02278735'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:21.423832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e39453776b423b91e9c90e02278735'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:21.424752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e39453776b423b91e9c90e02278735'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:21.444760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e39453776b423b91e9c90e02278735'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:21.455016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e39453776b423b91e9c90e02278735'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:21.475601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1036/2078] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:20 2022] DELETE /identity/v3/users/a2e39453776b423b91e9c90e02278735 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:21.477343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:21.477819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:21.478016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:21.498774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:21.500122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:21.576670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1043/2079] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:21.591933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:21.594264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:21.695146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238a8c5-93e0-4a57-8fe9-d35f23203c9c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:21.699269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1667da-d588-4a2f-8f5f-8c21d3e8f524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:21.699976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1667da-d588-4a2f-8f5f-8c21d3e8f524 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:21.700147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1667da-d588-4a2f-8f5f-8c21d3e8f524 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:21.700488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1667da-d588-4a2f-8f5f-8c21d3e8f524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:21.713234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1667da-d588-4a2f-8f5f-8c21d3e8f524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:21.714923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1667da-d588-4a2f-8f5f-8c21d3e8f524 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:21.727268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1037/2080] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:22.592449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:22.594714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:22.624013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b02c7-e889-452a-91b3-7aeaf94895cf None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:22.634014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:22.634906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:22.635332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:22.635747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] PATH_INFO: `/v3/projects/2456d44db4d444859f92e9b563c5c9dc` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:22.637703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2456d44db4d444859f92e9b563c5c9dc)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:22.641134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:22.702306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:22.859976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2456d44db4d444859f92e9b563c5c9dc', 'request_id': 'req-e0e9d590-1be9-49b6-8210-c049409effd1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:22.971949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2456d44db4d444859f92e9b563c5c9dc', 'request_id': 'req-e0e9d590-1be9-49b6-8210-c049409effd1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:22.989764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2456d44db4d444859f92e9b563c5c9dc', 'request_id': 'req-e0e9d590-1be9-49b6-8210-c049409effd1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:23.003394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] The token cache is being invalidate because project 2456d44db4d444859f92e9b563c5c9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:23.004161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:23.005289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e0e9d590-1be9-49b6-8210-c049409effd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:23.007453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1044/2081] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:22 2022] DELETE /identity/v3/projects/2456d44db4d444859f92e9b563c5c9dc => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:23.012757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.013260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.023185 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.039254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:23.041237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:23.140462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1038/2082] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:23.149168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.149584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.149870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.178126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:23.179511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:23.193129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1045/2083] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:23.197850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:23.217622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dda33-d6ec-4c1c-b480-e6187da6c273 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:23.229778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:23.230419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.230705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.230978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.231633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:23.232072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:23.239193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:23.264763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1039/2084] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:23.269307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:23.285212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1eda3-981e-4350-971a-f965d1bf9e66 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:23.297884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:23.297884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.297884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.297884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.298354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:23.299350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:23.432061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1046/2085] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:23 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:23.436099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:23.447800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760d5c7-bcfc-4f0f-9f7c-93d7b66ade45 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:23.463475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:23.526343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:23.527298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.527728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.528167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.529049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:23.531353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:23.702640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1040/2086] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:23 2022] POST /identity/v3/users => generated 286 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:23.704004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:23.716490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ae80a-1732-405d-b9d1-61a47ef0a09c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:23.722309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:23.723571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.724069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.724486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.725139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:23.726744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:23.740974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1047/2087] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:23.744705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:23.771757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301131b8-4e7c-49e9-b32a-82edf6620e1a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:23.779877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70a293-f62a-4d36-af03-9e5075b85a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:23.781210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70a293-f62a-4d36-af03-9e5075b85a3f admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.781210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70a293-f62a-4d36-af03-9e5075b85a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.781801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70a293-f62a-4d36-af03-9e5075b85a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.783535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70a293-f62a-4d36-af03-9e5075b85a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:23.786264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70a293-f62a-4d36-af03-9e5075b85a3f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:23.806883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1041/2088] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:23.811020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:23.824219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e63f6-64bc-49d1-9aa1-7409dc44a22c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:23.830795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:23.830795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.830795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.831283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] PATH_INFO: `/v3/projects/28f0cda364854dfd9fde0249a751ecf4/users/c639977b5f6744b9939d1e5a9745cd03/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.849502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f0cda364854dfd9fde0249a751ecf4, user_id=c639977b5f6744b9939d1e5a9745cd03, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:23.853194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:23.956424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1048/2089] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:23 2022] PUT /identity/v3/projects/28f0cda364854dfd9fde0249a751ecf4/users/c639977b5f6744b9939d1e5a9745cd03/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:23.958281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ae437-4f6c-4bff-8511-40cccf2492f0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:23.958964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ae437-4f6c-4bff-8511-40cccf2492f0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:23.959471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ae437-4f6c-4bff-8511-40cccf2492f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:23.987661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e96ae437-4f6c-4bff-8511-40cccf2492f0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:23.990433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e96ae437-4f6c-4bff-8511-40cccf2492f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:24.083772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1042/2090] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:24.091856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:24.094700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:24.217209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c194412-6f78-4ad5-b00e-5b083550a22f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:24.229978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a59adb-9162-42db-9e66-5a2e2271f7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:24.231192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a59adb-9162-42db-9e66-5a2e2271f7fa service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:24.231192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a59adb-9162-42db-9e66-5a2e2271f7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:24.231421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a59adb-9162-42db-9e66-5a2e2271f7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:24.263567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a59adb-9162-42db-9e66-5a2e2271f7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:24.265544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a59adb-9162-42db-9e66-5a2e2271f7fa service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:24.277629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1049/2091] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:25.890943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb479-7fae-4a3d-bc12-a03595e8aa28 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:25.891906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb479-7fae-4a3d-bc12-a03595e8aa28 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:25.892371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb479-7fae-4a3d-bc12-a03595e8aa28 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:25.894604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1043/2092] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:36:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:36:25.907427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16f530-7615-4c46-b580-90c2dbed06bf None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:25.908308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16f530-7615-4c46-b580-90c2dbed06bf None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:25.908892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16f530-7615-4c46-b580-90c2dbed06bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:25.957804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ce16f530-7615-4c46-b580-90c2dbed06bf None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:25.960008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ce16f530-7615-4c46-b580-90c2dbed06bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:26.061559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:26.077941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb479-7fae-4a3d-bc12-a03595e8aa28 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:26.080016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fd9eb479-7fae-4a3d-bc12-a03595e8aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:26.105496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1050/2093] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:36:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:26.126297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb479-7fae-4a3d-bc12-a03595e8aa28 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:26.126848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff947-c40f-42b2-b947-3818a090c9d4 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:26.126848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff947-c40f-42b2-b947-3818a090c9d4 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:26.126848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff947-c40f-42b2-b947-3818a090c9d4 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:26.129651 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:26.129994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1051/2094] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 31 00:36:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:36:26.130820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:26.131151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:26.131710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] PATH_INFO: `/v3/users/703b8e63fa4449dfb85de594a8d2e5fe` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:26.132978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=703b8e63fa4449dfb85de594a8d2e5fe)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:26.135289 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:26.135530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:26.136046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:26.136564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:26.192361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:26.193706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:26.208574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1052/2095] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 31 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:26.222754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:26.234636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a276fe-110c-4530-8c15-90df76a325fc None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:26.240810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:26.241438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:26.241716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:26.241988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:26.251471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:26.257630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:26.274688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1053/2096] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:36:26.361600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703b8e63fa4449dfb85de594a8d2e5fe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:26.452119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703b8e63fa4449dfb85de594a8d2e5fe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:26.453031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703b8e63fa4449dfb85de594a8d2e5fe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:26.500928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703b8e63fa4449dfb85de594a8d2e5fe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:26.509316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-dde2d609-0042-4778-8b76-f56860ec3f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703b8e63fa4449dfb85de594a8d2e5fe'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:26.519367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1044/2097] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:26 2022] DELETE /identity/v3/users/703b8e63fa4449dfb85de594a8d2e5fe => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:27.034176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:27.051520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40cd31-0191-4e55-b39b-358648e777f7 service placement] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:27.158457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:27.159378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.159764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.160811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] PATH_INFO: `/v3/projects/c0528602037241c98f36d12984aa26a9` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.162229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0528602037241c98f36d12984aa26a9)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:27.165080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:27.352256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0528602037241c98f36d12984aa26a9', 'request_id': 'req-67831dcf-5349-4b12-9a7b-ad5d7a922afa'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:27.467539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0528602037241c98f36d12984aa26a9', 'request_id': 'req-67831dcf-5349-4b12-9a7b-ad5d7a922afa'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:27.474747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0528602037241c98f36d12984aa26a9', 'request_id': 'req-67831dcf-5349-4b12-9a7b-ad5d7a922afa'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:27.491559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] The token cache is being invalidate because project c0528602037241c98f36d12984aa26a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:27.491883 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:27.492763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-67831dcf-5349-4b12-9a7b-ad5d7a922afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:27.494425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1054/2098] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:27 2022] DELETE /identity/v3/projects/c0528602037241c98f36d12984aa26a9 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:27.500077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.506955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.506955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.525971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:27.527128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:27.532460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:27.590852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1045/2099] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:27.598770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.599087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.599385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.621682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:27.621682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:27.629897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1055/2100] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:27.634263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:27.645237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a43a7-4f64-4b94-b4e9-9dcded6171d7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:27.651086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:27.651710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.651996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.652270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.653029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:27.653337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:27.655033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:27.669205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1046/2101] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:36:27.673557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:27.683870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cb6ca-c6e0-44a6-bc5b-e38521692e52 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:27.688821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:27.689504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.689781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.690531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.691357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:27.694021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:27.777225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1056/2102] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:27 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:27.778643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:27.792198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5528dc1-7088-4707-8767-e97473575da5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:27.855741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:27.856741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.857021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.857294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.857953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:27.859910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:27.959816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1047/2103] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:27 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:27.964438 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:27.980266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81cb3c-8425-4b05-81b7-baadd81a2c56 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:27.985716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:27.988815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:27.988815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:27.988815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:27.988815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:27.994622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:28.011046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1057/2104] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:28.015924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:28.042084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0adb9f-0019-4782-ad59-d47d07e895f3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:28.051761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b853448-ef5e-4cd8-ba32-779a6686ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:28.051761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b853448-ef5e-4cd8-ba32-779a6686ab83 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:28.051761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b853448-ef5e-4cd8-ba32-779a6686ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:28.051761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b853448-ef5e-4cd8-ba32-779a6686ab83 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:28.052123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b853448-ef5e-4cd8-ba32-779a6686ab83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:28.053806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b853448-ef5e-4cd8-ba32-779a6686ab83 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:28.076978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1048/2105] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:28.081746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:28.099861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608eb52-1f4b-4b5b-ada4-ebe52d848a21 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:28.105542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:28.106762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:28.107204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:28.109840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] PATH_INFO: `/v3/projects/5023455241594360bacff911a6641d7a/users/ac2476465c4c4bd2b1cf81934b6e9a7d/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:28.132025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5023455241594360bacff911a6641d7a, user_id=ac2476465c4c4bd2b1cf81934b6e9a7d, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:28.134734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:28.137659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:28.219384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1058/2106] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:28 2022] PUT /identity/v3/projects/5023455241594360bacff911a6641d7a/users/ac2476465c4c4bd2b1cf81934b6e9a7d/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:28.225362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:28.225656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:28.225932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:28.246709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:28.247954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:28.322963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1049/2107] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:28.333162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:28.335034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:28.502821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051493-9fda-40b6-ae4f-d178af44503b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:28.502821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa14cc-30d9-4a35-8630-fb41d39b5732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:28.502821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa14cc-30d9-4a35-8630-fb41d39b5732 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:28.502821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa14cc-30d9-4a35-8630-fb41d39b5732 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:28.503153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa14cc-30d9-4a35-8630-fb41d39b5732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:28.503153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa14cc-30d9-4a35-8630-fb41d39b5732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:28.503153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa14cc-30d9-4a35-8630-fb41d39b5732 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:28.516561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1059/2108] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:36:30.989560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:30.991711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:31.026411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fff8d4-40e4-4482-a520-80d47b924110 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:31.052092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:31.053763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:31.054611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:31.056162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] PATH_INFO: `/v3/users/05bc430fed28409b8f3aee71c0e6cc1b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:31.063735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05bc430fed28409b8f3aee71c0e6cc1b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:31.064740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:31.100892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:31.108894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:31.109491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:31.109580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:31.184212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] MFA Rules not processed for user `c639977b5f6744b9939d1e5a9745cd03`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:31.185509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:31.273837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1060/2109] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:31.278762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:31.282343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:31.309847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd82a3-4a52-4ac0-9366-0217d38a4071 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:31.320861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:31.321980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:31.322381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:31.322641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:31.324470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:31.330415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:31.365616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05bc430fed28409b8f3aee71c0e6cc1b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:31.451902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05bc430fed28409b8f3aee71c0e6cc1b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:31.452693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05bc430fed28409b8f3aee71c0e6cc1b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:31.490755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05bc430fed28409b8f3aee71c0e6cc1b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:31.497577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1061/2110] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:31 2022] POST /identity/v3/projects => generated 373 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:31.503029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:31.517340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-391a179a-937d-477c-bda8-e52d208d1e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05bc430fed28409b8f3aee71c0e6cc1b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:31.535023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1050/2111] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:30 2022] DELETE /identity/v3/users/05bc430fed28409b8f3aee71c0e6cc1b => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:31.541984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:31.542595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:31.543227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:31.609707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:31.613183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:31.690455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a561a-99d3-409e-8d87-c4894a05b111 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:31.706196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:31.768035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1051/2112] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:31.783841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:31.784052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:31.820113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:31.820886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:31.821233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:31.821793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:31.822494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:31.824596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:31.887617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126479f-9f70-4911-8263-663d46d43e58 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:31.892683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:31.893393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:31.893723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:31.893998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:31.905794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:31.907459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:31.920376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1052/2113] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:31.997002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1062/2114] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:31 2022] POST /identity/v3/users => generated 293 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:31.998702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:32.001400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:32.042789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1a62c-eade-46d2-83a4-5d762e594f83 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:32.064808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:32.066698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:32.067299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:32.068158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:32.069095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:32.078156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:32.118356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:32.119287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1053/2115] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:32.150189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb1362-22c6-4cfb-998d-1468d9dbe4cc admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:32.158804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:32.159503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:32.160732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:32.161002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] PATH_INFO: `/v3/projects/45c50656acc7484aa36ceb69fcd61f83/users/84bb2bd64d1348109d85415eda58f2c3/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:32.182538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c50656acc7484aa36ceb69fcd61f83, user_id=84bb2bd64d1348109d85415eda58f2c3, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:32.185509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:32.286134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1063/2116] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:32 2022] PUT /identity/v3/projects/45c50656acc7484aa36ceb69fcd61f83/users/84bb2bd64d1348109d85415eda58f2c3/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:32.290328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:32.319907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c612-ed73-46f1-9a87-5372fc618cbe admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:32.362201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:32.449743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:32.459393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:32.459393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:32.459393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:32.459393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:32.461051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:32.500077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:32.505857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1054/2117] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:32.523366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629fb89-5fd1-4d32-884b-2fc89c13651f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:32.533824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f489e-6b88-464e-b4d5-f6088b4e9be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:32.534606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f489e-6b88-464e-b4d5-f6088b4e9be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:32.535674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f489e-6b88-464e-b4d5-f6088b4e9be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:32.536082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f489e-6b88-464e-b4d5-f6088b4e9be0 admin admin] PATH_INFO: `/v3/projects/45c50656acc7484aa36ceb69fcd61f83/users/84bb2bd64d1348109d85415eda58f2c3/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:32.538716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f489e-6b88-464e-b4d5-f6088b4e9be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c50656acc7484aa36ceb69fcd61f83, user_id=84bb2bd64d1348109d85415eda58f2c3, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:32.541657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f489e-6b88-464e-b4d5-f6088b4e9be0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:32.562355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1064/2118] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:32 2022] PUT /identity/v3/projects/45c50656acc7484aa36ceb69fcd61f83/users/84bb2bd64d1348109d85415eda58f2c3/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:32.761446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:32.761446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:32.774913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da677-40e5-4120-a92c-7d7c340bdc4c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:32.888249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:32.889924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:32.890423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:32.891421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] PATH_INFO: `/v3/projects/2d1582f5fce54f96b6c641f79b6d5833` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:32.895505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1582f5fce54f96b6c641f79b6d5833)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:32.898965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:33.140938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1582f5fce54f96b6c641f79b6d5833', 'request_id': 'req-6d723541-8a27-4c6f-aacc-8f2ae326c54b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:33.149390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1582f5fce54f96b6c641f79b6d5833', 'request_id': 'req-6d723541-8a27-4c6f-aacc-8f2ae326c54b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:33.156638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1582f5fce54f96b6c641f79b6d5833', 'request_id': 'req-6d723541-8a27-4c6f-aacc-8f2ae326c54b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:33.306223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] The token cache is being invalidate because project 2d1582f5fce54f96b6c641f79b6d5833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:33.306517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:33.307269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6d723541-8a27-4c6f-aacc-8f2ae326c54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:33.309000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1055/2119] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:32 2022] DELETE /identity/v3/projects/2d1582f5fce54f96b6c641f79b6d5833 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:33.314560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:33.315394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:33.315670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:33.343542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:33.346405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:33.431955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1065/2120] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:33.441244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:33.441535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:33.441813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:33.465329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:33.468557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:33.486824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1056/2121] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:33.492125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:33.513103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5e6a3-550f-4fca-a371-ddd2342f66b1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:33.528450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:33.529083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:33.529355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:33.532142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:33.532142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:33.532142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:33.533776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:33.557498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1066/2122] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:33.567022 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:33.583047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52499cc6-c68a-4fcd-992a-5e39cd6922fa None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:33.588612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:33.589489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:33.589913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:33.590329 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:33.591539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:33.594008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:33.690592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1057/2123] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:33 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:33.695253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:33.708076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e309-f8e4-4d2e-bd5f-e1a048512c48 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:33.795619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:33.824534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:33.825270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:33.825612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:33.826007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:33.837972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:33.840479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:33.960344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1067/2124] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:33 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:36:33.964611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:33.980466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16962c01-e14a-4e60-aa2a-731a834b340c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:33.996568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:33.997380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:33.997665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:33.997941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:33.998802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:34.001600 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:34.042496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1058/2125] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:34.044157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:34.060804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab5219-2eb6-483b-8178-15554d4d625f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:34.076361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23670ba1-3368-499f-afbc-530e57a1fe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:34.076782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23670ba1-3368-499f-afbc-530e57a1fe5b admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:34.077080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23670ba1-3368-499f-afbc-530e57a1fe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:34.077354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23670ba1-3368-499f-afbc-530e57a1fe5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:34.078025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23670ba1-3368-499f-afbc-530e57a1fe5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:34.079644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23670ba1-3368-499f-afbc-530e57a1fe5b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:34.106682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1068/2126] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:34.111846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:34.137250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfad0d-787b-4a6c-9e54-2b4bd7d07ead admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:34.152695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:34.154090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:34.154556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:34.155056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] PATH_INFO: `/v3/projects/7355b09a086648fbab8c6eb8c9954b7f/users/e9fad7886e8a4e74ae135b215bf51f52/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:34.189892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7355b09a086648fbab8c6eb8c9954b7f, user_id=e9fad7886e8a4e74ae135b215bf51f52, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:34.196047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:34.320904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d3467-802d-4a19-90b7-0fa0a223203b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:34.321613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1059/2127] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:34 2022] PUT /identity/v3/projects/7355b09a086648fbab8c6eb8c9954b7f/users/e9fad7886e8a4e74ae135b215bf51f52/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:34.322244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d3467-802d-4a19-90b7-0fa0a223203b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:34.322575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d3467-802d-4a19-90b7-0fa0a223203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:34.372063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:34.373403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ad7d3467-802d-4a19-90b7-0fa0a223203b None None] MFA Rules not processed for user `e9fad7886e8a4e74ae135b215bf51f52`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:34.374705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ad7d3467-802d-4a19-90b7-0fa0a223203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:34.512846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1069/2128] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:34.526033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:34.529123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:34.636839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07e257-ae6c-4965-855c-d74b3a44aa06 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:34.645547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:34.646173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:34.646606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:34.647279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:34.670257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:34.674160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:34.701359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1060/2129] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:38.910983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3198-c95f-422a-99ca-a588f363e005 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:38.911890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3198-c95f-422a-99ca-a588f363e005 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:38.914099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3198-c95f-422a-99ca-a588f363e005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:38.976796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-738e3198-c95f-422a-99ca-a588f363e005 None None] MFA Rules not processed for user `ac2476465c4c4bd2b1cf81934b6e9a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:38.978603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-738e3198-c95f-422a-99ca-a588f363e005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:39.092166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1070/2130] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:39.106637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:39.108190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:39.200232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c68c8-f0c1-4c69-b04e-9e446bab0981 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:39.207002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:39.207966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:39.209271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:39.209271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:39.224213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:39.226168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:39.237986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1061/2131] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:42.681086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a38a1-fb9e-44a1-accd-5e6eccc12303 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:42.681679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a38a1-fb9e-44a1-accd-5e6eccc12303 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:42.681679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a38a1-fb9e-44a1-accd-5e6eccc12303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:42.715481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-790a38a1-fb9e-44a1-accd-5e6eccc12303 None None] MFA Rules not processed for user `84bb2bd64d1348109d85415eda58f2c3`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:42.717191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-790a38a1-fb9e-44a1-accd-5e6eccc12303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:42.771472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:42.809319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1071/2132] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:42.827674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:42.829734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:42.858676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba46e-a7ba-4f03-8385-2062f29b7a60 service nova] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:42.867482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:42.868858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:42.869403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:42.869844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:42.873533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:42.975768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:42.977124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:42.987544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1062/2133] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:43.034497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13039d-6bd1-49c8-acc6-c6deb62c367a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:43.035240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13039d-6bd1-49c8-acc6-c6deb62c367a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:43.035501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13039d-6bd1-49c8-acc6-c6deb62c367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:43.057892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5c13039d-6bd1-49c8-acc6-c6deb62c367a None None] MFA Rules not processed for user `ac2476465c4c4bd2b1cf81934b6e9a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:43.059113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5c13039d-6bd1-49c8-acc6-c6deb62c367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:43.067378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1072/2134] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:44.368894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:44.371024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:44.405219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:44.523380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c5347-8180-4751-b968-36fbf1df3d8a service nova] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:44.564231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:44.564986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:44.566564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:44.567464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] PATH_INFO: `/v3/users/5280b583fec14939b5a77c7198230e23` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:44.570869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5280b583fec14939b5a77c7198230e23)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:44.575335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:44.867519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5280b583fec14939b5a77c7198230e23'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:44.958249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5280b583fec14939b5a77c7198230e23'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:44.964358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5280b583fec14939b5a77c7198230e23'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:44.978857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5280b583fec14939b5a77c7198230e23'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:44.992843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5280b583fec14939b5a77c7198230e23'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:44.998563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1063/2135] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:44 2022] DELETE /identity/v3/users/5280b583fec14939b5a77c7198230e23 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:45.004091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:45.004635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:45.004635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:45.033825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:45.035825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:45.141020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1073/2136] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:45.153006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:45.154544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:45.297189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14162de8-df1e-44b7-8a8a-85188c7d6725 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:45.301784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff1998-a0bd-46b3-ae45-2aacac99bc10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:45.303031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff1998-a0bd-46b3-ae45-2aacac99bc10 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:45.303307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff1998-a0bd-46b3-ae45-2aacac99bc10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:45.303594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff1998-a0bd-46b3-ae45-2aacac99bc10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:45.314063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff1998-a0bd-46b3-ae45-2aacac99bc10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:45.315546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff1998-a0bd-46b3-ae45-2aacac99bc10 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:45.341867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1064/2137] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:46.048995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:46.051243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:46.074955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdae06-72e6-4710-9888-bf957a80862b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:46.087072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:46.087811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:46.088540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:46.088540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] PATH_INFO: `/v3/projects/fe2e4e87e5d9414f8a5001d2705a1aa0` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:46.094181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2e4e87e5d9414f8a5001d2705a1aa0)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:46.096245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:46.214367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2e4e87e5d9414f8a5001d2705a1aa0', 'request_id': 'req-2ac8a88d-e529-461a-8be1-e6b678db2226'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.242065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:46.242777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:46.242777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:46.271544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] MFA Rules not processed for user `c639977b5f6744b9939d1e5a9745cd03`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:46.273156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:46.307005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:46.314358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2e4e87e5d9414f8a5001d2705a1aa0', 'request_id': 'req-2ac8a88d-e529-461a-8be1-e6b678db2226'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.321914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2e4e87e5d9414f8a5001d2705a1aa0', 'request_id': 'req-2ac8a88d-e529-461a-8be1-e6b678db2226'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.345350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] The token cache is being invalidate because project fe2e4e87e5d9414f8a5001d2705a1aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:46.345893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.347734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.351553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1074/2138] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:46 2022] DELETE /identity/v3/projects/fe2e4e87e5d9414f8a5001d2705a1aa0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:46.361301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1065/2139] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:46.362048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:46.381143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:46.473810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:46.519694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a88d-e529-461a-8be1-e6b678db2226 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:46.529625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:46.530400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:46.530804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:46.531012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] PATH_INFO: `/v3/users/afa7fe025e494bd4a46c236debc89d75` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:46.532437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa7fe025e494bd4a46c236debc89d75)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:46.535603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:46.730286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa7fe025e494bd4a46c236debc89d75'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.743712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa7fe025e494bd4a46c236debc89d75'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.755751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa7fe025e494bd4a46c236debc89d75'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.839933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa7fe025e494bd4a46c236debc89d75'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.840619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a257fff6-642a-48f1-be1a-89304c8064e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa7fe025e494bd4a46c236debc89d75'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:46.847893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1075/2140] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:46 2022] DELETE /identity/v3/users/afa7fe025e494bd4a46c236debc89d75 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:47.458103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:47.474541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725fa9e-f54a-42d6-89a4-037054e91500 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:47.571563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:47.572212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:47.572577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:47.572885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] PATH_INFO: `/v3/projects/2098701f7e9245909ebe2b53e057a8fb` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:47.574084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2098701f7e9245909ebe2b53e057a8fb)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:47.576160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:47.713585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2098701f7e9245909ebe2b53e057a8fb', 'request_id': 'req-3b56551e-b826-47dd-871e-3581a15369ef'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:47.720701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2098701f7e9245909ebe2b53e057a8fb', 'request_id': 'req-3b56551e-b826-47dd-871e-3581a15369ef'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:47.729925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2098701f7e9245909ebe2b53e057a8fb', 'request_id': 'req-3b56551e-b826-47dd-871e-3581a15369ef'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:47.796003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] The token cache is being invalidate because project 2098701f7e9245909ebe2b53e057a8fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:47.796449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:47.797149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3b56551e-b826-47dd-871e-3581a15369ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:47.798678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1066/2141] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:47 2022] DELETE /identity/v3/projects/2098701f7e9245909ebe2b53e057a8fb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:47.804550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:47.805694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:47.805694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:47.827650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:47.828906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:47.897426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1076/2142] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:47.905441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:47.905746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:47.906017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:47.931008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:47.932243 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:47.947398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1067/2143] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:47.952120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:47.962578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ec52-440a-4ad9-bfba-08684abd150f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:47.971168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:47.971168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:47.971168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:47.971168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:47.971588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:47.971588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:47.971588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:47.985005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1077/2144] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:47.990197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:47.999861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ab7b-0868-4072-b458-402b193d69ec None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:48.006039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:48.007064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.007348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.007618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.008511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:48.010664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:48.130447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1068/2145] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:47 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:48.133632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:48.147903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddeedd-37d3-4a1a-95c3-79041695f254 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:48.228701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:48.235504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:48.236121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.236435 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.236707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.237421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:48.239669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:48.341270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1078/2146] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:48 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:48.344709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:48.358965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765668-7bd9-42a5-87e3-2fcdf80745a6 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:48.367353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:48.370025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.370538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.370852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.371582 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:48.373346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:48.411537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1069/2147] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:48.419503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:48.430573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52948802-5f20-4d14-8a1f-ef95e21ac7cb admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:48.438096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a1d1b-afdf-45d2-a737-09104a0f12fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:48.438443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a1d1b-afdf-45d2-a737-09104a0f12fc admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.438727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a1d1b-afdf-45d2-a737-09104a0f12fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.438997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a1d1b-afdf-45d2-a737-09104a0f12fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.439718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a1d1b-afdf-45d2-a737-09104a0f12fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:48.441872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a1d1b-afdf-45d2-a737-09104a0f12fc admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:48.453980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1079/2148] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:48.458923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:48.477905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b548b-faf5-4968-a4ae-55fbf1e42dc3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:48.484396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:48.485919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.486358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.488571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] PATH_INFO: `/v3/projects/ec60dc08741f49958ebb97f8fcd0e46b/users/1fc991adcb2a41cd952daa04012af728/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.514731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec60dc08741f49958ebb97f8fcd0e46b, user_id=1fc991adcb2a41cd952daa04012af728, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:48.518126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:48.616295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1070/2149] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:48 2022] PUT /identity/v3/projects/ec60dc08741f49958ebb97f8fcd0e46b/users/1fc991adcb2a41cd952daa04012af728/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:48.628188 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.628543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.628858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.666993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] MFA Rules not processed for user `1fc991adcb2a41cd952daa04012af728`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:48.668782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:48.749733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1080/2150] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:48.754907 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:48.764110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:48.830731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f2c3-ef73-4523-8cfd-fcba9cdafa42 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:48.840357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:48.841002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:48.841283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:48.841574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:48.842397 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:48.844597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:48.949715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1071/2151] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:48 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:36:48.954416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:48.966273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ccbeb-c844-4a24-88af-9f5fc35cba1a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:49.041778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:49.042445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.042726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.042999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.043718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:49.045759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:49.164648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1081/2152] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:48 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:49.165940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:49.178712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fadb8-5709-4f61-ab37-da854b24f21f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:49.185262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:49.185799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.186078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.186387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.187034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:49.188982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:49.209725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1072/2153] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:49.213660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:49.231081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ff682-56a3-4f7c-b90a-ef48dd45d346 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:49.250527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:49.251291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.254392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.254392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] PATH_INFO: `/v3/projects/f097a11a973f41519cc846333ac8e513/users/4ee74eedcb3d41efa28b5285d15393e2/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.279881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f097a11a973f41519cc846333ac8e513, user_id=4ee74eedcb3d41efa28b5285d15393e2, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:49.283258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:49.356301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1082/2154] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:49 2022] PUT /identity/v3/projects/f097a11a973f41519cc846333ac8e513/users/4ee74eedcb3d41efa28b5285d15393e2/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:49.360560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:49.373746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11925796-826f-4f16-ab53-2a29be2b80d9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:49.472860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2968f-1e46-4cd1-82e2-8c82ce7680c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:49.473882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2968f-1e46-4cd1-82e2-8c82ce7680c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.474401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2968f-1e46-4cd1-82e2-8c82ce7680c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.474793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2968f-1e46-4cd1-82e2-8c82ce7680c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.475515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2968f-1e46-4cd1-82e2-8c82ce7680c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:49.477278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2968f-1e46-4cd1-82e2-8c82ce7680c0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:49.489760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1073/2155] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:49.494120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:49.504949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147c65-f279-4f23-832f-177b5a196416 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:49.509757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:49.510415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.510695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.511075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] PATH_INFO: `/v3/projects/f097a11a973f41519cc846333ac8e513/users/4ee74eedcb3d41efa28b5285d15393e2/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.513024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f097a11a973f41519cc846333ac8e513, user_id=4ee74eedcb3d41efa28b5285d15393e2, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:49.515687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:49.529566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1083/2156] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:49 2022] PUT /identity/v3/projects/f097a11a973f41519cc846333ac8e513/users/4ee74eedcb3d41efa28b5285d15393e2/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:49.541361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.541361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.543688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.574172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] MFA Rules not processed for user `4ee74eedcb3d41efa28b5285d15393e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:49.575407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:49.665941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:49.683887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1074/2157] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:49.709615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:49.711388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:49.748082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:49.797071 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345fc93-4187-46e2-92d5-0368629d641b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:49.801491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:49.802234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:49.802516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:49.802791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:49.812927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:49.814440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:49.828406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1084/2158] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:51.406321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:51.430694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1ca1-5477-4bda-845b-443ba2c8e7c7 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:51.437232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee74eedcb3d41efa28b5285d15393e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f097a11a973f41519cc846333ac8e513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:51.438030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:51.438408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:51.438988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:51.439824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:51.440156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:51.442364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:51.455881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1075/2159] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:51.460897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:51.471232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3f6c-215f-4127-b737-80741512486e service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:51.477065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee74eedcb3d41efa28b5285d15393e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f097a11a973f41519cc846333ac8e513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:51.477696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:51.478120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:51.478363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:51.480597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:51.487261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:51.580511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1085/2160] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:51 2022] POST /identity/v3/projects => generated 352 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:51.587394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:51.599569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219278d-8f5f-43ad-a2b1-55d2cb5ab190 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:51.677470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee74eedcb3d41efa28b5285d15393e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f097a11a973f41519cc846333ac8e513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:51.678320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:51.678602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:51.678912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:51.679607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:51.679913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:51.681837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:51.701140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1076/2161] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:51.745901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:51.757898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6231ef8-b464-415a-b64f-2c4eedc6f06c tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:51.770674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee74eedcb3d41efa28b5285d15393e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f097a11a973f41519cc846333ac8e513', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:51.770674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:51.770674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:51.771163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] PATH_INFO: `/v3/projects/07d0df2f65674110858147383d038dc0` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:51.771163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorizing `identity:delete_project(project_id=07d0df2f65674110858147383d038dc0)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:51.771163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:51.898251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d0df2f65674110858147383d038dc0', 'request_id': 'req-0ea6333a-5bac-47e8-82da-e3840ca45319'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:51.983344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d0df2f65674110858147383d038dc0', 'request_id': 'req-0ea6333a-5bac-47e8-82da-e3840ca45319'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:51.991946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d0df2f65674110858147383d038dc0', 'request_id': 'req-0ea6333a-5bac-47e8-82da-e3840ca45319'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:52.012457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] The token cache is being invalidate because project 07d0df2f65674110858147383d038dc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:52.012766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:52.013421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0ea6333a-5bac-47e8-82da-e3840ca45319 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:52.014952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1086/2162] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:51 2022] DELETE /identity/v3/projects/07d0df2f65674110858147383d038dc0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:52.808084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:52.810424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:52.891714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517b031-e0ca-4763-99e1-6b33f5a48ed7 tempest-VolumeQuotaClassesTest-1937902688 tempest-VolumeQuotaClassesTest-1937902688-project-admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:52.974235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:52.975107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:52.975552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:52.976156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] PATH_INFO: `/v3/users/1fc991adcb2a41cd952daa04012af728` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:52.977955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc991adcb2a41cd952daa04012af728)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:52.981306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:53.083124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:53.184236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc991adcb2a41cd952daa04012af728'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:53.275029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc991adcb2a41cd952daa04012af728'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:53.275965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc991adcb2a41cd952daa04012af728'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:53.293284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc991adcb2a41cd952daa04012af728'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:53.302088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc991adcb2a41cd952daa04012af728'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:53.309146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1077/2163] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:52 2022] DELETE /identity/v3/users/1fc991adcb2a41cd952daa04012af728 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:53.313363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a55fbf-bdb6-4b8f-a9e9-98bd626b3964 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:53.313859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a55fbf-bdb6-4b8f-a9e9-98bd626b3964 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:53.314060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a55fbf-bdb6-4b8f-a9e9-98bd626b3964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:53.335807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-03a55fbf-bdb6-4b8f-a9e9-98bd626b3964 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:53.335807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-03a55fbf-bdb6-4b8f-a9e9-98bd626b3964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:53.410730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1087/2164] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:53.420859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:53.422604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:53.499169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdfaef-af5c-4fa3-b32e-19f30eaefab8 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:53.505729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:53.506356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:53.506634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:53.506899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:53.516838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:53.518300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:53.530212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1078/2165] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:53.739755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:53.740219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:53.740599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:53.762376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] MFA Rules not processed for user `022c4688d9ce4df9a99ae5fa84e1d871`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:53.763735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:53.785366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:53.838908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1088/2166] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:54.233323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:54.237077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:54.248671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742fa8c-f178-4e5b-a38e-78d26ca2e29d service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:54.253844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:54.254624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:54.254943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:54.255328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] PATH_INFO: `/v3/projects/ec60dc08741f49958ebb97f8fcd0e46b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:54.257445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec60dc08741f49958ebb97f8fcd0e46b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:54.259779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:54.398505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec60dc08741f49958ebb97f8fcd0e46b', 'request_id': 'req-28f46c5e-3311-48c9-b2d0-73614dd21f82'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.404770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec60dc08741f49958ebb97f8fcd0e46b', 'request_id': 'req-28f46c5e-3311-48c9-b2d0-73614dd21f82'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.423513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec60dc08741f49958ebb97f8fcd0e46b', 'request_id': 'req-28f46c5e-3311-48c9-b2d0-73614dd21f82'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.557698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] The token cache is being invalidate because project ec60dc08741f49958ebb97f8fcd0e46b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:54.558039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.558705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.560638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1079/2167] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:54 2022] DELETE /identity/v3/projects/ec60dc08741f49958ebb97f8fcd0e46b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:54.565155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:54.577626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33204a01-2f8c-4e9d-96f1-1f2474c895e3 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:54.662467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:54.663044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:54.663409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:54.663832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] PATH_INFO: `/v3/users/4ee74eedcb3d41efa28b5285d15393e2` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:54.665574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee74eedcb3d41efa28b5285d15393e2)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:54.668120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:54.890031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee74eedcb3d41efa28b5285d15393e2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.920771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee74eedcb3d41efa28b5285d15393e2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:54.930999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee74eedcb3d41efa28b5285d15393e2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.028888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee74eedcb3d41efa28b5285d15393e2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.029519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ad9d4e2d-22e7-4013-a642-bb8538bc4cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee74eedcb3d41efa28b5285d15393e2'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.037868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1089/2168] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:54 2022] DELETE /identity/v3/users/4ee74eedcb3d41efa28b5285d15393e2 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:55.550880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:55.552993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:55.568316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f46c5e-3311-48c9-b2d0-73614dd21f82 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:55.604693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:55.636944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:55.637590 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:55.637869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:55.638190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] PATH_INFO: `/v3/projects/f097a11a973f41519cc846333ac8e513` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:55.639418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f097a11a973f41519cc846333ac8e513)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:55.641395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:55.817598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f097a11a973f41519cc846333ac8e513', 'request_id': 'req-17c6f4d6-a6f6-4fa7-9076-25c89e006829'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.825657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f097a11a973f41519cc846333ac8e513', 'request_id': 'req-17c6f4d6-a6f6-4fa7-9076-25c89e006829'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.837055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f097a11a973f41519cc846333ac8e513', 'request_id': 'req-17c6f4d6-a6f6-4fa7-9076-25c89e006829'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.926093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] The token cache is being invalidate because project f097a11a973f41519cc846333ac8e513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:55.926447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.928502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-17c6f4d6-a6f6-4fa7-9076-25c89e006829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:55.930808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1080/2169] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:55 2022] DELETE /identity/v3/projects/f097a11a973f41519cc846333ac8e513 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:55.938601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:55.939206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:55.939206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:55.971683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:55.973283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:56.037231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:56.068787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1090/2170] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:56.070975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.071443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.071718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.094551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:56.095814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:56.105121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1081/2171] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:56.109500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.126312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0218d-7d97-4f44-9656-5c2c10fce237 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.133515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.134314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.134667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.135010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.135935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:56.140768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:56.140768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:56.166453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1091/2172] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:36:56.170338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.178929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b532-2a26-4e47-b775-1ac90a259907 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.185483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.186288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.186571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.186847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.187662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:56.189599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:56.313895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1082/2173] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:56 2022] POST /identity/v3/projects => generated 361 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:56.315236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.325747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f31f7-8711-44db-8762-db431079692e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.391373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.392044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.392332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.392664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.393301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:56.395859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:56.546982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1092/2174] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:56 2022] POST /identity/v3/users => generated 281 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:56.550856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.562334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31618fc-8cde-4861-bccc-e9dcf3d92145 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.568668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.568668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.569028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.569169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.569828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:56.571707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:56.584082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1083/2175] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:56.592703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.604135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c251c2-da18-4231-bbfa-9525598936e4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.609153 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394006-19cd-48e7-9877-97f6ba619910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.611346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394006-19cd-48e7-9877-97f6ba619910 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.611628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394006-19cd-48e7-9877-97f6ba619910 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.611917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394006-19cd-48e7-9877-97f6ba619910 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.612600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394006-19cd-48e7-9877-97f6ba619910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:56.614553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394006-19cd-48e7-9877-97f6ba619910 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:56.626596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1093/2176] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:56.631795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.645934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1522-66f2-48f1-b6ce-bc9a0e2faa64 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.650902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.651554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.651836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.652113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] PATH_INFO: `/v3/projects/5a6e09e6b46448d1b6f072c06c046057/users/8f57ca7f53de446ca8b0aaf153c505e1/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.671647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6e09e6b46448d1b6f072c06c046057, user_id=8f57ca7f53de446ca8b0aaf153c505e1, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:56.674443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:56.762570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1084/2177] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:56 2022] PUT /identity/v3/projects/5a6e09e6b46448d1b6f072c06c046057/users/8f57ca7f53de446ca8b0aaf153c505e1/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:56.775126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.775410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.775682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.807173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] MFA Rules not processed for user `8f57ca7f53de446ca8b0aaf153c505e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:56.808501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:56.878676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1094/2178] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:56.914997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:56.916765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:56.982548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f564e-ce3e-48c8-ad0e-52ea33a4d218 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:56.986602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146386-f24a-4cdc-a6f7-333216e04332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:56.987990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146386-f24a-4cdc-a6f7-333216e04332 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:56.988408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146386-f24a-4cdc-a6f7-333216e04332 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:56.988838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146386-f24a-4cdc-a6f7-333216e04332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:56.998867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84146386-f24a-4cdc-a6f7-333216e04332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:57.000636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84146386-f24a-4cdc-a6f7-333216e04332 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:57.006831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:57.011992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1085/2179] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:57.034289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:57.093141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:57.117275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf6989-dc83-42b9-bc04-a7cad2bbc91d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:57.122710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:57.129718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:57.129718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:57.129718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] PATH_INFO: `/v3/users/8f57ca7f53de446ca8b0aaf153c505e1` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:57.136378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f57ca7f53de446ca8b0aaf153c505e1)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:57.138437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:57.348434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f57ca7f53de446ca8b0aaf153c505e1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:57.363681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f57ca7f53de446ca8b0aaf153c505e1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:57.371175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f57ca7f53de446ca8b0aaf153c505e1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:57.446317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f57ca7f53de446ca8b0aaf153c505e1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:57.447910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f57ca7f53de446ca8b0aaf153c505e1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:57.462199 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1095/2180] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:36:57 2022] DELETE /identity/v3/users/8f57ca7f53de446ca8b0aaf153c505e1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:57.467163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:57.467370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:57.467737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:57.492866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:57.503648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:57.562470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1086/2181] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:57.574950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:57.576752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:57.643305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33b4d1-9ee9-423b-9110-981ab88c85df admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:57.648949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34be315-e0f6-40b2-b721-e7dd8ad3f281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:57.649799 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be315-e0f6-40b2-b721-e7dd8ad3f281 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:57.650286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be315-e0f6-40b2-b721-e7dd8ad3f281 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:57.650763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be315-e0f6-40b2-b721-e7dd8ad3f281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:57.666909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34be315-e0f6-40b2-b721-e7dd8ad3f281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:57.668568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34be315-e0f6-40b2-b721-e7dd8ad3f281 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:57.680846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1096/2182] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:58.219636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:58.222959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:58.236787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa275f3-d4e0-410a-8eea-9f33425f5837 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:58.241881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:58.242523 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:58.242804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:58.243083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] PATH_INFO: `/v3/projects/5a6e09e6b46448d1b6f072c06c046057` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:58.244436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6e09e6b46448d1b6f072c06c046057)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:58.246425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:58.355451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6e09e6b46448d1b6f072c06c046057', 'request_id': 'req-afd801e6-94e1-4446-8226-763316fcd434'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:58.361556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6e09e6b46448d1b6f072c06c046057', 'request_id': 'req-afd801e6-94e1-4446-8226-763316fcd434'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:58.371246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6e09e6b46448d1b6f072c06c046057', 'request_id': 'req-afd801e6-94e1-4446-8226-763316fcd434'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:58.455610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] The token cache is being invalidate because project 5a6e09e6b46448d1b6f072c06c046057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:36:58.455610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:58.456905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-afd801e6-94e1-4446-8226-763316fcd434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:36:58.469964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1087/2183] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:36:58 2022] DELETE /identity/v3/projects/5a6e09e6b46448d1b6f072c06c046057 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:58.470278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:58.470278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:58.470278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:58.494341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:58.511670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:58.607900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1097/2184] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:58.615858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:58.616840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:58.617343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:58.643173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:58.645722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:58.661950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1088/2185] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:36:58.666822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:58.674788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:58.678218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa1b96-4752-447f-81d4-3b3e262c4805 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:58.683718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:58.684641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:58.684853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:58.685242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:58.685958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:36:58.686335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:58.688298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:58.706837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1098/2186] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:36:58.712400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:58.723174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca051cc-11ef-4f3c-a116-7744ca44bff2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:58.728895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:58.730203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:58.731401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:58.731791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:58.732479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:58.734421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:58.806362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1089/2187] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:58 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:36:58.809695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:58.823922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d5ec0-6c3b-4479-9ccf-62c450bdc9ac admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:58.897303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:58.897919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:58.898198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:58.898481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:58.899114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:58.901138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.002452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1099/2188] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:58 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:59.008066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.018565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746530-b525-4cd1-b03b-c85c64178237 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.024418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.025163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.025440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.025729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.026565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.032727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.044072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1090/2189] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:36:59.049253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.072182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7eb91-5eaf-4a91-a142-53b550622ba8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.081300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847651cc-85c5-4235-a15e-b23389601327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.082055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847651cc-85c5-4235-a15e-b23389601327 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.082423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847651cc-85c5-4235-a15e-b23389601327 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.082654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847651cc-85c5-4235-a15e-b23389601327 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.083427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847651cc-85c5-4235-a15e-b23389601327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.085233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847651cc-85c5-4235-a15e-b23389601327 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.099154 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1100/2190] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:59.104270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.117607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3369be-f2ee-4b05-b23b-3275eea7d5ec admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.129541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.130615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.131118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.132821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] PATH_INFO: `/v3/projects/4f8579b74a89465080df8cfe47771bb4/users/c7b72439bc1743f98d73cdb1ac860e2b/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.151291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8579b74a89465080df8cfe47771bb4, user_id=c7b72439bc1743f98d73cdb1ac860e2b, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.154570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.232790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1091/2191] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:59 2022] PUT /identity/v3/projects/4f8579b74a89465080df8cfe47771bb4/users/c7b72439bc1743f98d73cdb1ac860e2b/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:36:59.240654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.241175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.241670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.286839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] MFA Rules not processed for user `c7b72439bc1743f98d73cdb1ac860e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:36:59.288549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:59.376882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1101/2192] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:36:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:36:59.383548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.412180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:59.466991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f386fc-8538-4a4d-8e3d-a0f1fa2036ad admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.471978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.472688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.473142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.473403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.474041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.476090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.586488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1092/2193] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:36:59 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:36:59.590765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.603546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd74ba4-9255-44b1-8eb2-1bed4ced4118 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.675793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.676301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.676742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.677648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.678292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.682468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.691940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:59.766851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1102/2194] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:36:59 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:36:59.771458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.788207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aa3e5-ddbe-4977-8cdc-1f83d3800d6f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.796058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.797504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.798010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.798382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.799133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.802035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.814180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1093/2195] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:36:59.822284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.833693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ac83-89f3-4182-926c-575eeebd46d9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:59.840914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:59.841777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:59.843885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:59.843885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] PATH_INFO: `/v3/projects/728825fcb620416b8186b6904654051d/users/b8f32845f03545d1b19e8b02595edbcb/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:59.859120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=728825fcb620416b8186b6904654051d, user_id=b8f32845f03545d1b19e8b02595edbcb, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:59.861979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:59.939412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1103/2196] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:36:59 2022] PUT /identity/v3/projects/728825fcb620416b8186b6904654051d/users/b8f32845f03545d1b19e8b02595edbcb/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:36:59.944128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:59.958077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd884eac-0f0d-43a0-b8df-cf452cd963af admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:00.034915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afa8c5-3937-4b8a-a0b9-cc43e66e7678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:00.038794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afa8c5-3937-4b8a-a0b9-cc43e66e7678 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:00.038794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afa8c5-3937-4b8a-a0b9-cc43e66e7678 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:00.038794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afa8c5-3937-4b8a-a0b9-cc43e66e7678 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:00.038794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afa8c5-3937-4b8a-a0b9-cc43e66e7678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:00.039819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afa8c5-3937-4b8a-a0b9-cc43e66e7678 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:00.056355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1094/2197] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:00.058382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:00.089970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf9341-b0b8-49e7-b208-c123f4487ffc admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:00.089970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:00.089970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:00.089970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:00.090325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] PATH_INFO: `/v3/projects/728825fcb620416b8186b6904654051d/users/b8f32845f03545d1b19e8b02595edbcb/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:00.090325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728825fcb620416b8186b6904654051d, user_id=b8f32845f03545d1b19e8b02595edbcb, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:00.090325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:00.095539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1104/2198] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:00 2022] PUT /identity/v3/projects/728825fcb620416b8186b6904654051d/users/b8f32845f03545d1b19e8b02595edbcb/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:00.106627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:00.106916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:00.107193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:00.138710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] MFA Rules not processed for user `b8f32845f03545d1b19e8b02595edbcb`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:00.139943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:00.221713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1095/2199] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:00.235793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:00.238841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:00.360953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177080e-be83-496b-8178-709e4993f823 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:00.367392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491746a-920c-4900-a480-fb2d631c75ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:00.368572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491746a-920c-4900-a480-fb2d631c75ec service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:00.368572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491746a-920c-4900-a480-fb2d631c75ec service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:00.368754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491746a-920c-4900-a480-fb2d631c75ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:00.442596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491746a-920c-4900-a480-fb2d631c75ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:00.444497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491746a-920c-4900-a480-fb2d631c75ec service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:00.454977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1105/2200] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:03.439325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:03.459696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:03.514310 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc255c07-5a8f-4a23-a9e4-bec84ddc33d9 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:03.518795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:03.519432 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:03.519710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:03.519977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] PATH_INFO: `/v3/users/c7b72439bc1743f98d73cdb1ac860e2b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:03.521748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b72439bc1743f98d73cdb1ac860e2b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:03.524724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:03.709923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b72439bc1743f98d73cdb1ac860e2b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:03.800753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b72439bc1743f98d73cdb1ac860e2b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:03.801386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b72439bc1743f98d73cdb1ac860e2b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:03.816385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b72439bc1743f98d73cdb1ac860e2b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:03.823628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b72439bc1743f98d73cdb1ac860e2b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:03.830647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1096/2201] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:03 2022] DELETE /identity/v3/users/c7b72439bc1743f98d73cdb1ac860e2b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:03.835286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:03.835811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:03.835811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:03.864969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:03.866187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:03.938351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1106/2202] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:03.948971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:03.950256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:04.030614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616d1e7-6f7e-4fbc-91ef-69e6008d8363 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:04.035342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:04.036092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:04.036392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:04.036666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:04.056199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:04.056738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:04.074348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1097/2203] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:04.360259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:04.363274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:04.380547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e1732-4daf-4024-86f3-ea8d0c9ae8c8 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:04.386600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:04.387539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:04.392482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:04.392482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] PATH_INFO: `/v3/users/e9fad7886e8a4e74ae135b215bf51f52` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:04.392482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fad7886e8a4e74ae135b215bf51f52)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:04.392482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:04.395672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:04.651214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fad7886e8a4e74ae135b215bf51f52'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.680427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fad7886e8a4e74ae135b215bf51f52'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.680427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fad7886e8a4e74ae135b215bf51f52'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.686337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:04.688447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:04.703906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261817b3-68f6-4041-90a6-4b42aa48b64e service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:04.711208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:04.712001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:04.712482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:04.713577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] PATH_INFO: `/v3/projects/4f8579b74a89465080df8cfe47771bb4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:04.714017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8579b74a89465080df8cfe47771bb4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:04.716366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:04.760849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fad7886e8a4e74ae135b215bf51f52'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.761654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ba3a13b0-ae16-454d-bfe8-b3d9574a5ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fad7886e8a4e74ae135b215bf51f52'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.768166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1107/2204] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:04 2022] DELETE /identity/v3/users/e9fad7886e8a4e74ae135b215bf51f52 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:04.774040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:04.774332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:04.774670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:04.795360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:04.796629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:04.837974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:04.865671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8579b74a89465080df8cfe47771bb4', 'request_id': 'req-1b627a85-0d31-4435-8728-9e5a764acb98'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.868282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1108/2205] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:04.875115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8579b74a89465080df8cfe47771bb4', 'request_id': 'req-1b627a85-0d31-4435-8728-9e5a764acb98'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.880031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:04.882033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:04.882811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8579b74a89465080df8cfe47771bb4', 'request_id': 'req-1b627a85-0d31-4435-8728-9e5a764acb98'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.971149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e4c3a-c350-4ca0-a692-e853925d01e9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:04.991283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] The token cache is being invalidate because project 4f8579b74a89465080df8cfe47771bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:04.991644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.992684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:04.994433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1098/2206] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:04 2022] DELETE /identity/v3/projects/4f8579b74a89465080df8cfe47771bb4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:37:04.998520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:05.000237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:05.026145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:05.045172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:05.046065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:05.046344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:05.046610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:05.048459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:05.074575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b627a85-0d31-4435-8728-9e5a764acb98 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:05.088369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:05.089406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:05.089854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:05.090802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] PATH_INFO: `/v3/users/b8f32845f03545d1b19e8b02595edbcb` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:05.092711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8f32845f03545d1b19e8b02595edbcb)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:05.097794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:05.142306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:05.143656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:05.156778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1109/2207] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:05.309242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8f32845f03545d1b19e8b02595edbcb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:05.372243 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8f32845f03545d1b19e8b02595edbcb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:05.373592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8f32845f03545d1b19e8b02595edbcb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:05.412215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8f32845f03545d1b19e8b02595edbcb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:05.426046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5938aa77-60a4-46b3-a5e6-8bf3ec9bb6a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8f32845f03545d1b19e8b02595edbcb'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:05.432699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1099/2208] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:04 2022] DELETE /identity/v3/users/b8f32845f03545d1b19e8b02595edbcb => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:37:05.901818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:05.904000 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:05.918547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2630aa-f896-4370-83cf-112e7d86a78a service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:05.983322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:05.983963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:05.984238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:05.984553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] PATH_INFO: `/v3/projects/7355b09a086648fbab8c6eb8c9954b7f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:05.985678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7355b09a086648fbab8c6eb8c9954b7f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:05.988117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.079721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7355b09a086648fbab8c6eb8c9954b7f', 'request_id': 'req-b9799bb8-c49c-4382-a0a6-9818c3afab35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.137148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7355b09a086648fbab8c6eb8c9954b7f', 'request_id': 'req-b9799bb8-c49c-4382-a0a6-9818c3afab35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.143493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7355b09a086648fbab8c6eb8c9954b7f', 'request_id': 'req-b9799bb8-c49c-4382-a0a6-9818c3afab35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.155667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] The token cache is being invalidate because project 7355b09a086648fbab8c6eb8c9954b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:06.155952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.156631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.158333 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1110/2209] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:05 2022] DELETE /identity/v3/projects/7355b09a086648fbab8c6eb8c9954b7f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:06.164222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b5f88-3594-4d32-9ab8-d6d3dc9348eb None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.164697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b5f88-3594-4d32-9ab8-d6d3dc9348eb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.167690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b5f88-3594-4d32-9ab8-d6d3dc9348eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.168573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.170697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:06.190979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:06.210895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e79b5f88-3594-4d32-9ab8-d6d3dc9348eb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:06.211986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e79b5f88-3594-4d32-9ab8-d6d3dc9348eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:06.250475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9799bb8-c49c-4382-a0a6-9818c3afab35 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.253750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:06.257840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.258529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.258809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.259556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] PATH_INFO: `/v3/projects/728825fcb620416b8186b6904654051d` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.260741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=728825fcb620416b8186b6904654051d)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.262857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.282376 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1100/2210] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:06.292082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.292576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.292853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.315300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:06.316654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:06.325241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1101/2211] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:06.329679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.340018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505ded4-dbbf-4eb8-9fc3-8642192d24d2 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.344896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.347873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.347873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.347873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.347873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:37:06.347873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.349632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.364604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1102/2212] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:37:06.369603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.380895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef66a3-e013-43ab-9fea-079bc919e55b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.386426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728825fcb620416b8186b6904654051d', 'request_id': 'req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.444535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.445155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.445430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.445697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.446330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.448254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.458933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728825fcb620416b8186b6904654051d', 'request_id': 'req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.465173 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728825fcb620416b8186b6904654051d', 'request_id': 'req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.488890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] The token cache is being invalidate because project 728825fcb620416b8186b6904654051d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:06.488890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.494999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6a7e69eb-a10d-4914-8f3c-2a379b8cd56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:06.496094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1111/2213] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:06 2022] DELETE /identity/v3/projects/728825fcb620416b8186b6904654051d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:37:06.498963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb769c6-47c9-43ea-afdc-bec4caec85cc None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.498963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb769c6-47c9-43ea-afdc-bec4caec85cc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.498963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb769c6-47c9-43ea-afdc-bec4caec85cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.536918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2eb769c6-47c9-43ea-afdc-bec4caec85cc None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:06.538256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2eb769c6-47c9-43ea-afdc-bec4caec85cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:06.551611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1112/2214] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:06.559570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.559815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.560091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.583463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:06.585379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:06.602247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1103/2215] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:37:06.604474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1113/2216] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:06.606516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.609395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:06.609529 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.619332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dd002-08f0-41b0-8fc6-491bd89a33f1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.687119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787911a-d2d9-4c4e-9e99-458dd8727822 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.689288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.690073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.690314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.690868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.692791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.692946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:37:06.692946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.693295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.693727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.693929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.694559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.694769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.696609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.718217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1114/2217] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:37:06.719480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.731401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b4f7e-d300-4ef3-94fb-82fbafc3a9e9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.736114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.736798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.737078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.737345 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.738001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.741341 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.777989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1104/2218] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:06.782186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.792652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3e27-18a5-48b7-9126-b41818957f85 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.798189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.798801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.799078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.799486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.800668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.801856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.820307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1105/2219] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:06.824188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.835514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d5c7f-6bbf-43f4-a098-fa2dc639dcf8 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.840459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.841164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.841508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.841861 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.842591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.844253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.855792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1106/2220] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:37:06.859697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.875412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554ec1c-3f5e-424c-b385-98d54bd237c9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.876142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1115/2221] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:06.877507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:06.883822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:06.887492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:06.945571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5171e-4833-4abf-b7b5-113276eccd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.946258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5171e-4833-4abf-b7b5-113276eccd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.949536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5171e-4833-4abf-b7b5-113276eccd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.949536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5171e-4833-4abf-b7b5-113276eccd84 admin admin] PATH_INFO: `/v3/projects/3e02f07bd0b041188bc7c7bd31a7ae68/users/56694042782c43f591d6d8ddef5c0350/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.957806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033de9cb-c604-4b0e-a78e-399e759ba762 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:06.963827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:06.965131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:06.965415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:06.965688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:06.966344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.968474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5171e-4833-4abf-b7b5-113276eccd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e02f07bd0b041188bc7c7bd31a7ae68, user_id=56694042782c43f591d6d8ddef5c0350, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:06.972565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5171e-4833-4abf-b7b5-113276eccd84 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:06.981182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.055053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1107/2222] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:06 2022] PUT /identity/v3/projects/3e02f07bd0b041188bc7c7bd31a7ae68/users/56694042782c43f591d6d8ddef5c0350/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:37:07.083572 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1116/2223] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:37:06 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:07.086193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.086563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.087164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.087991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.108196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6befd5a-dd01-4d05-83f6-8ce110e8bb69 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.149318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] MFA Rules not processed for user `56694042782c43f591d6d8ddef5c0350`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:07.151509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:07.218630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.219260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.221533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.221533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.221533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.223335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.238526 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1108/2224] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:07.240872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1117/2225] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:07.243072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.246055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.258260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d124f-0b7d-459d-87d6-9995b882ae6a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.262581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafc93-d0c8-48a2-82f1-acb1dea7ebcc admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.264736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.265437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.265784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.266229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.267069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.273373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.286126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.287214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.287492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.287747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.290420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.292477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.308281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1118/2226] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:07.312516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.327152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbd6cb-235a-4aae-a930-adae406697a8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.332801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266768c2-6f38-466b-b6dd-510db25640cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.333456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266768c2-6f38-466b-b6dd-510db25640cf admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.333737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266768c2-6f38-466b-b6dd-510db25640cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.334012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266768c2-6f38-466b-b6dd-510db25640cf admin admin] PATH_INFO: `/v3/projects/d83682ef35a54057ad44a83594c20460/users/7f8e7037e334469e848864af07f69c5b/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.357745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266768c2-6f38-466b-b6dd-510db25640cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83682ef35a54057ad44a83594c20460, user_id=7f8e7037e334469e848864af07f69c5b, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.363063 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266768c2-6f38-466b-b6dd-510db25640cf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.369285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1109/2227] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:37:07 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:37:07.373794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.473444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47379026-87f5-4b5b-b1f1-3a90870e9814 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.483395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.488792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.489478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.489691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1119/2228] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:07 2022] PUT /identity/v3/projects/d83682ef35a54057ad44a83594c20460/users/7f8e7037e334469e848864af07f69c5b/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:37:07.490125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.490986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.493052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.501714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.502006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.502425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.505253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:07.539437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] MFA Rules not processed for user `7f8e7037e334469e848864af07f69c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:07.540849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:07.601350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1110/2229] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:37:07 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:07.605346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.616161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1120/2230] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:07.617034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567bc97-f7a3-451f-bc7d-5d491f0cc176 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.628582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.630117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:07.716497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.717126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.717404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.717678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.718328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.719391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8bc2c-5eda-4434-b673-c9ab0ea18997 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.720034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.726721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.727449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.727907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.728252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.736551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1111/2231] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:37:07.739515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.744102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.745664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.754208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9599ae-0844-41a4-a47d-afa0c51256de admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.759178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1121/2232] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:37:07.760125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5027-6229-4667-809c-c03d181835a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.761024 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5027-6229-4667-809c-c03d181835a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.761119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5027-6229-4667-809c-c03d181835a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.761469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5027-6229-4667-809c-c03d181835a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.762175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5027-6229-4667-809c-c03d181835a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.764065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5027-6229-4667-809c-c03d181835a4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.778324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1112/2233] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:07.782618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:07.805259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ec62d-72d9-4b56-bbb7-0d10bfe643f2 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:07.810503 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:07.811347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.811629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.811956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] PATH_INFO: `/v3/projects/9f996a1cc7c24b3996fbd13c2c5fc3d6/users/f7215c7c90db4d08a98f0ce9a44e6356/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:07.835281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f996a1cc7c24b3996fbd13c2c5fc3d6, user_id=f7215c7c90db4d08a98f0ce9a44e6356, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:07.839293 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:07.853073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:07.916297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1122/2234] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:07 2022] PUT /identity/v3/projects/9f996a1cc7c24b3996fbd13c2c5fc3d6/users/f7215c7c90db4d08a98f0ce9a44e6356/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:07.931120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:07.934008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:07.934363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:08.000182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] MFA Rules not processed for user `f7215c7c90db4d08a98f0ce9a44e6356`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:08.001805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:08.152724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1113/2235] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:08.155877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:08.281952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77a53-076a-4a72-80bd-305f742e71d3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:08.288519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:08.289168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:08.289458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:08.289746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:08.290374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:08.296797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:08.496372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1123/2236] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:37:08 2022] POST /identity/v3/projects => generated 385 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:08.500280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:08.518085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8c8e1-6b3a-419b-8462-be1894790f0c None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:08.586772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:08.587418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:08.587752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:08.588031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:08.589063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:08.590664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:08.704409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:08.711509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1114/2237] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:37:08 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:08.715786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:08.741514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042774e2-95e6-41a7-9006-f4a851e79aa1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:08.751164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:08.751791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:08.752072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:08.752366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:08.753049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:08.755824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:08.768958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1124/2238] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:08.773793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:08.790612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dbe4a-f04c-407a-b296-e99cdc314eb1 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:08.802804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:08.805199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:08.810146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:08.810476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] PATH_INFO: `/v3/projects/13953299c4ed4fe9b3ccc35bdf25e610/users/c8b9693b71ce4c0f91856d2e672f5186/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:08.836862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13953299c4ed4fe9b3ccc35bdf25e610, user_id=c8b9693b71ce4c0f91856d2e672f5186, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:08.839704 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:08.917111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1115/2239] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:08 2022] PUT /identity/v3/projects/13953299c4ed4fe9b3ccc35bdf25e610/users/c8b9693b71ce4c0f91856d2e672f5186/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:08.921302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:08.933638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70608d0-12f6-41d4-b0e5-69535e5d68a1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:08.976147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:09.001520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c763f-d1be-4e2b-aa19-acd9b253067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:09.002233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c763f-d1be-4e2b-aa19-acd9b253067a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:09.004985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c763f-d1be-4e2b-aa19-acd9b253067a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:09.004985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c763f-d1be-4e2b-aa19-acd9b253067a admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:09.004985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c763f-d1be-4e2b-aa19-acd9b253067a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:09.015002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c763f-d1be-4e2b-aa19-acd9b253067a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:09.033882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1125/2240] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:37:09.038971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:09.051446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fb7d1-8500-4460-8e2a-26459654922a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:09.058593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:09.059512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:09.060007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:09.060515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] PATH_INFO: `/v3/projects/13953299c4ed4fe9b3ccc35bdf25e610/users/c8b9693b71ce4c0f91856d2e672f5186/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:09.062985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=13953299c4ed4fe9b3ccc35bdf25e610, user_id=c8b9693b71ce4c0f91856d2e672f5186, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:09.065963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:09.116356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1116/2241] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:09 2022] PUT /identity/v3/projects/13953299c4ed4fe9b3ccc35bdf25e610/users/c8b9693b71ce4c0f91856d2e672f5186/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:37:09.116938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:09.116938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:09.116938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:09.146054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] MFA Rules not processed for user `c8b9693b71ce4c0f91856d2e672f5186`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:09.148208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:09.230796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1126/2242] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:09.243074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:09.303166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c57330-9408-4f02-83dd-a3674860c6ae admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:09.307051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:09.307835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:09.308041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:09.308313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:09.374215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:09.375686 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:09.385918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1117/2243] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:11.508848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:11.518773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123a20a-c0f8-4c98-b4cb-07b3672aa6e1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:11.523651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65000925-9057-4c56-8871-9abd9c932dc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:11.524349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65000925-9057-4c56-8871-9abd9c932dc8 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:11.524658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65000925-9057-4c56-8871-9abd9c932dc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:11.524924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65000925-9057-4c56-8871-9abd9c932dc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:11.586244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65000925-9057-4c56-8871-9abd9c932dc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:11.587312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65000925-9057-4c56-8871-9abd9c932dc8 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:11.596649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1127/2244] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:12.800990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:12.877261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6c52-d9cd-4768-b715-c9335fad5fc7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:12.877801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6c52-d9cd-4768-b715-c9335fad5fc7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:12.877801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc6c52-d9cd-4768-b715-c9335fad5fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:12.900046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a6cc6c52-d9cd-4768-b715-c9335fad5fc7 None None] MFA Rules not processed for user `77d23c99308c414f939761b07d329e03`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:12.900422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77556f37-7bd2-43e3-97cb-faa2b4b8b226 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:12.901342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a6cc6c52-d9cd-4768-b715-c9335fad5fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:12.905640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:12.906272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:12.906551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:12.906857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] PATH_INFO: `/v3/users/7f8e7037e334469e848864af07f69c5b` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:12.908032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8e7037e334469e848864af07f69c5b)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:12.910367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:12.927098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:12.966516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1128/2245] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:13.136643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8e7037e334469e848864af07f69c5b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:13.217941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8e7037e334469e848864af07f69c5b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:13.218967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8e7037e334469e848864af07f69c5b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:13.241745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8e7037e334469e848864af07f69c5b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:13.252842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8e7037e334469e848864af07f69c5b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:13.260230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1118/2246] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:12 2022] DELETE /identity/v3/users/7f8e7037e334469e848864af07f69c5b => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:13.264880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:13.265273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:13.265668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:13.273925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:13.289391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:13.290702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:13.344996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:13.346448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:13.376952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1129/2247] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:13.403185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:13.406736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:13.423791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488fb0d-8215-46aa-8743-91808b621813 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:13.429901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:13.430599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:13.431009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:13.435253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:13.435253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:13.516132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc7f0f-f57a-4e34-a5ae-de163566f139 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:13.521107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00ea7c-f26a-405b-bf91-60fb42dd3b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:13.526011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00ea7c-f26a-405b-bf91-60fb42dd3b8f service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:13.526301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00ea7c-f26a-405b-bf91-60fb42dd3b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:13.526742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00ea7c-f26a-405b-bf91-60fb42dd3b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:13.528644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:13.532918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:13.546307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00ea7c-f26a-405b-bf91-60fb42dd3b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:13.547855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00ea7c-f26a-405b-bf91-60fb42dd3b8f service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:13.548767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1119/2248] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:37:13.575934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1130/2249] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:37:14.163293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:14.165572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:14.220606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115538-d839-4f6d-abb9-aa631cd8be67 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:14.227312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:14.227981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:14.228243 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:14.228535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] PATH_INFO: `/v3/projects/d83682ef35a54057ad44a83594c20460` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:14.232192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83682ef35a54057ad44a83594c20460)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:14.234196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:14.366401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:14.373645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83682ef35a54057ad44a83594c20460', 'request_id': 'req-221f075a-39c1-40b7-94b7-72079421a2ab'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:14.381044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83682ef35a54057ad44a83594c20460', 'request_id': 'req-221f075a-39c1-40b7-94b7-72079421a2ab'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:14.387844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83682ef35a54057ad44a83594c20460', 'request_id': 'req-221f075a-39c1-40b7-94b7-72079421a2ab'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:14.511061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] The token cache is being invalidate because project d83682ef35a54057ad44a83594c20460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:14.512696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:14.512696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-221f075a-39c1-40b7-94b7-72079421a2ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:14.514182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1120/2250] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:14 2022] DELETE /identity/v3/projects/d83682ef35a54057ad44a83594c20460 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:14.519144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:14.520267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:14.520713 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:14.541118 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:14.542468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:14.624609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1131/2251] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:14.632766 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:14.633069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:14.633318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:14.661884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:14.664129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:14.672385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1121/2252] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:14.677280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:14.687555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ac3d-665d-40a3-ad17-e71fd75dab6b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:14.694110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:14.694866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:14.695225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:14.695588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:14.696309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:37:14.696850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:14.698520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:14.711968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1132/2253] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:37:14.716530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:14.731970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf74daf-5d2d-4fa9-9566-5b4a1a2694e8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:14.737113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:14.738424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:14.742051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:14.742051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:14.742051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:14.742051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:14.819400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1122/2254] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:37:14 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:37:14.823735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:14.834705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451277a0-9077-4916-a2fd-d4c635a66854 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:14.867894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:14.901540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:14.902207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:14.902483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:14.902753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:14.903386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:14.905352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:15.001895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1133/2255] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:37:14 2022] POST /identity/v3/users => generated 282 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:37:15.011560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:15.041682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffdb8f-b865-4689-a7bd-ae5559e7dea9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:15.066986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:15.067679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:15.068724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:15.068724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:15.069070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:15.070741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:15.093039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1123/2256] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:15.097134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:15.117534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e9999-eb0e-4945-8139-986a3c0de1fb admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:15.127500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1b2d4-be5e-4160-8089-ff58c3c49518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:15.128289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1b2d4-be5e-4160-8089-ff58c3c49518 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:15.129308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1b2d4-be5e-4160-8089-ff58c3c49518 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:15.129910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1b2d4-be5e-4160-8089-ff58c3c49518 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:15.130911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1b2d4-be5e-4160-8089-ff58c3c49518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:15.132973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1b2d4-be5e-4160-8089-ff58c3c49518 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:15.156877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1134/2257] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:37:15.159548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:15.171973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d8f76-7d9f-432d-803d-96d34a3cc35e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:15.190520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:15.191748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:15.192038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:15.192314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] PATH_INFO: `/v3/projects/9ce59252a5704781848f9f54640e674e/users/7200fea87a8f411980e8280d654e3f7f/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:15.212456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce59252a5704781848f9f54640e674e, user_id=7200fea87a8f411980e8280d654e3f7f, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:15.215059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:15.298494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1124/2258] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:37:15 2022] PUT /identity/v3/projects/9ce59252a5704781848f9f54640e674e/users/7200fea87a8f411980e8280d654e3f7f/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:15.309027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:15.309428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:15.309696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:15.344714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] MFA Rules not processed for user `7200fea87a8f411980e8280d654e3f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:15.346283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:15.429157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1135/2259] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:15.442173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:15.444012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:15.504910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508ecb1-47ca-4aac-b558-cce2bddd1a08 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:15.508736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cb56d-6c9e-43d7-bf9c-79351dae600e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:15.509335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cb56d-6c9e-43d7-bf9c-79351dae600e service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:15.509610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cb56d-6c9e-43d7-bf9c-79351dae600e service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:15.509883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cb56d-6c9e-43d7-bf9c-79351dae600e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:15.518713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cb56d-6c9e-43d7-bf9c-79351dae600e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:15.520713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cb56d-6c9e-43d7-bf9c-79351dae600e service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:15.530504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1125/2260] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:18.515016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:18.517445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:18.612256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359bca0-5750-4ec9-9831-abedd86e80a1 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:18.618967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:18.619623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:18.619904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:18.620193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] PATH_INFO: `/v3/users/56694042782c43f591d6d8ddef5c0350` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:18.621383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56694042782c43f591d6d8ddef5c0350)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:18.623359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:18.792602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:18.795216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56694042782c43f591d6d8ddef5c0350'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:18.825144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56694042782c43f591d6d8ddef5c0350'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:18.835902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56694042782c43f591d6d8ddef5c0350'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:18.914613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56694042782c43f591d6d8ddef5c0350'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:18.915464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56694042782c43f591d6d8ddef5c0350'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:18.922700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1136/2261] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:18 2022] DELETE /identity/v3/users/56694042782c43f591d6d8ddef5c0350 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:18.927625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:18.928091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:18.929300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:18.968278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:18.972089 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:19.052412 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1126/2262] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:19.068546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:19.070298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:19.131632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66707874-ad8a-4122-a202-b67431709fc6 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:19.135475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:19.136192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:19.136571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:19.136941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:19.148301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:19.149852 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:19.165323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1137/2263] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:37:19.724707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:19.727288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:19.744308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0cac1-321e-4adc-834a-0d4b70fc1006 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:19.750187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:19.750859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:19.751936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:19.752142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] PATH_INFO: `/v3/projects/3e02f07bd0b041188bc7c7bd31a7ae68` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:19.753615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e02f07bd0b041188bc7c7bd31a7ae68)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:19.756171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:19.757961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:19.911711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e02f07bd0b041188bc7c7bd31a7ae68', 'request_id': 'req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:19.919294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e02f07bd0b041188bc7c7bd31a7ae68', 'request_id': 'req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:19.929554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e02f07bd0b041188bc7c7bd31a7ae68', 'request_id': 'req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.071444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] The token cache is being invalidate because project 3e02f07bd0b041188bc7c7bd31a7ae68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:20.071929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.075216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.077216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1127/2264] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:19 2022] DELETE /identity/v3/projects/3e02f07bd0b041188bc7c7bd31a7ae68 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:20.080988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:20.091668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a3329-7c7c-400f-a8c8-575e376dce4a service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:20.257226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:20.258124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:20.258413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:20.258735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] PATH_INFO: `/v3/users/f7215c7c90db4d08a98f0ce9a44e6356` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:20.260019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7215c7c90db4d08a98f0ce9a44e6356)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:20.262886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:20.546889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7215c7c90db4d08a98f0ce9a44e6356'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.560285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7215c7c90db4d08a98f0ce9a44e6356'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.563449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:20.571811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7215c7c90db4d08a98f0ce9a44e6356'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.674811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7215c7c90db4d08a98f0ce9a44e6356'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.675564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7215c7c90db4d08a98f0ce9a44e6356'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:20.682322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1138/2265] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:20 2022] DELETE /identity/v3/users/f7215c7c90db4d08a98f0ce9a44e6356 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:21.587115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:21.588767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:21.610963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1ed91-dda7-4c7c-b5e6-c79a308e8e68 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:21.687127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:21.688026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:21.688551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:21.689241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] PATH_INFO: `/v3/projects/9f996a1cc7c24b3996fbd13c2c5fc3d6` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:21.691087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f996a1cc7c24b3996fbd13c2c5fc3d6)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:21.694953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:21.791781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f996a1cc7c24b3996fbd13c2c5fc3d6', 'request_id': 'req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:21.797802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f996a1cc7c24b3996fbd13c2c5fc3d6', 'request_id': 'req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:21.805392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f996a1cc7c24b3996fbd13c2c5fc3d6', 'request_id': 'req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:21.851358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] The token cache is being invalidate because project 9f996a1cc7c24b3996fbd13c2c5fc3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:21.851829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:21.852268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:21.853790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1128/2266] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:21 2022] DELETE /identity/v3/projects/9f996a1cc7c24b3996fbd13c2c5fc3d6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:21.859219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:21.861012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:21.874631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25eb1-1b94-43c0-a930-e40fbbf98452 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:21.937097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:21.940308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:21.940308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:21.940308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] PATH_INFO: `/v3/users/c8b9693b71ce4c0f91856d2e672f5186` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:21.940308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b9693b71ce4c0f91856d2e672f5186)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:21.941884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:22.211857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b9693b71ce4c0f91856d2e672f5186'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:22.226640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b9693b71ce4c0f91856d2e672f5186'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:22.234901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b9693b71ce4c0f91856d2e672f5186'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:22.315336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b9693b71ce4c0f91856d2e672f5186'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:22.319382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b9693b71ce4c0f91856d2e672f5186'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:22.327268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1139/2267] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:37:21 2022] DELETE /identity/v3/users/c8b9693b71ce4c0f91856d2e672f5186 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:23.019108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:23.021384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:23.040574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffb6a0-e5eb-44f2-9caa-613893ce09e2 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:23.080566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:23.141010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:23.141675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:23.141948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:23.142257 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] PATH_INFO: `/v3/projects/13953299c4ed4fe9b3ccc35bdf25e610` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:23.143415 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13953299c4ed4fe9b3ccc35bdf25e610)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:23.146253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:23.260937 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13953299c4ed4fe9b3ccc35bdf25e610', 'request_id': 'req-f6d13d5a-0af0-4161-9ad9-617d924bae87'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:23.282431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13953299c4ed4fe9b3ccc35bdf25e610', 'request_id': 'req-f6d13d5a-0af0-4161-9ad9-617d924bae87'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:23.297381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13953299c4ed4fe9b3ccc35bdf25e610', 'request_id': 'req-f6d13d5a-0af0-4161-9ad9-617d924bae87'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:23.376265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] The token cache is being invalidate because project 13953299c4ed4fe9b3ccc35bdf25e610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:37:23.377433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:23.379748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-f6d13d5a-0af0-4161-9ad9-617d924bae87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:37:23.383691 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1129/2268] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:37:23 2022] DELETE /identity/v3/projects/13953299c4ed4fe9b3ccc35bdf25e610 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:37:23.930637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:23.932691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:24.011361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987827a-9a12-4ddc-9809-2696bda118c1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:24.033758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:24.078580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:24.079237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:24.079518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:24.079792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:24.081590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:24.165671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:24.167158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc6a38-edbe-4c7f-a065-a898b3c8239a service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:24.178070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1140/2269] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:24.716423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:24.716944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:24.717057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:24.738270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] MFA Rules not processed for user `45422a67a78f4784abc0f18ad84778f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:24.740118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:24.809003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1130/2270] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:37:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:34.801797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:34.949658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=42 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:35.423906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:35.424195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:35.424542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:35.451405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:35.455109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:35.579230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1141/2271] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:37:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:35.634565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:35.636991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:35.774999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49816fef-eede-4670-97d2-18266c0908e4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:35.780188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f6075-e1dc-4721-823a-341d748eb56f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:35.780837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f6075-e1dc-4721-823a-341d748eb56f service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:35.781119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f6075-e1dc-4721-823a-341d748eb56f service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:35.781396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f6075-e1dc-4721-823a-341d748eb56f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:35.792378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f6075-e1dc-4721-823a-341d748eb56f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:35.794496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f6075-e1dc-4721-823a-341d748eb56f service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:35.814228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1131/2272] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:37:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:37:40.252853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:40.254392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:40.267185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fbf97-8c00-4d33-8bb9-d6eac772acb9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:40.271172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:40.272052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:40.272457 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:40.273089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:40.275044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:40.346357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:40.347967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c063990-fd3a-45fb-9d91-5ab71674cf00 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:40.359312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1142/2273] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:45.810163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=32 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:50.353964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=55 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:54.804620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc2354-6cf8-40a3-916c-d03da0414f09 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:54.805247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc2354-6cf8-40a3-916c-d03da0414f09 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:54.805247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc2354-6cf8-40a3-916c-d03da0414f09 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:54.806569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1132/2274] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:37:54.812732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe2e3-e8aa-498c-a23f-ceb0aad91d76 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:54.813245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe2e3-e8aa-498c-a23f-ceb0aad91d76 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:54.813372 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe2e3-e8aa-498c-a23f-ceb0aad91d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:54.835178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fb7fe2e3-e8aa-498c-a23f-ceb0aad91d76 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:54.838050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fb7fe2e3-e8aa-498c-a23f-ceb0aad91d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:54.924437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1143/2275] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:37:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:37:54.937575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc2354-6cf8-40a3-916c-d03da0414f09 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:54.939699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4bfc2354-6cf8-40a3-916c-d03da0414f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:55.001572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc2354-6cf8-40a3-916c-d03da0414f09 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:55.005139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba655ec5-dc39-4dbc-8966-78704dc3b44b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:55.005735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba655ec5-dc39-4dbc-8966-78704dc3b44b service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.006009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba655ec5-dc39-4dbc-8966-78704dc3b44b service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.006276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba655ec5-dc39-4dbc-8966-78704dc3b44b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.016762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba655ec5-dc39-4dbc-8966-78704dc3b44b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:55.018570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba655ec5-dc39-4dbc-8966-78704dc3b44b service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:55.029897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1133/2276] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:55.236747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20d60-a9db-4c04-b4c1-b137f4c62679 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.237029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20d60-a9db-4c04-b4c1-b137f4c62679 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.237297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20d60-a9db-4c04-b4c1-b137f4c62679 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.238638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1144/2277] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:37:55.245699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aa74c-7dca-4f0a-96ed-032f35c66562 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.245995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aa74c-7dca-4f0a-96ed-032f35c66562 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.246250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aa74c-7dca-4f0a-96ed-032f35c66562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.265761 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-914aa74c-7dca-4f0a-96ed-032f35c66562 None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:55.266969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-914aa74c-7dca-4f0a-96ed-032f35c66562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:55.275087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1134/2278] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:37:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:55.287346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e20d60-a9db-4c04-b4c1-b137f4c62679 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:55.289802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-64e20d60-a9db-4c04-b4c1-b137f4c62679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:55.303322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e20d60-a9db-4c04-b4c1-b137f4c62679 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:55.307279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ada3f-294b-4f38-9c08-7e0aafca9193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:55.307820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ada3f-294b-4f38-9c08-7e0aafca9193 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.308099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ada3f-294b-4f38-9c08-7e0aafca9193 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.308404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ada3f-294b-4f38-9c08-7e0aafca9193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.315477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ada3f-294b-4f38-9c08-7e0aafca9193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:55.317019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ada3f-294b-4f38-9c08-7e0aafca9193 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:55.325124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1145/2279] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:55.411401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79869-0ee7-4cbb-b387-3d0d61aa9462 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.411720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79869-0ee7-4cbb-b387-3d0d61aa9462 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.411978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79869-0ee7-4cbb-b387-3d0d61aa9462 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.413986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1135/2280] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:37:55.419287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.420705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.420705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.440937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:37:55.442789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:55.452684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1146/2281] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:37:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:37:55.463934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d79869-0ee7-4cbb-b387-3d0d61aa9462 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:55.477515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d79869-0ee7-4cbb-b387-3d0d61aa9462 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:55.481501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68273d2-b74c-482f-adc4-5ab1ec45def3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:55.482311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68273d2-b74c-482f-adc4-5ab1ec45def3 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:55.482588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68273d2-b74c-482f-adc4-5ab1ec45def3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:55.482883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68273d2-b74c-482f-adc4-5ab1ec45def3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:55.494334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68273d2-b74c-482f-adc4-5ab1ec45def3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:55.495662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68273d2-b74c-482f-adc4-5ab1ec45def3 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:55.507595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1136/2282] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:59.041865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:59.043533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.118384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbfac4-bc6d-4951-8636-04af64c4264b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:59.123710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:59.124339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:59.124638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:59.124914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:59.126690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.139168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:59.140903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:59.151380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1147/2283] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:38:00.816190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:00.817274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:00.817729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:00.839015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:00.840234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:00.847716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1137/2284] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:38:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:00.859533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:00.870893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208bab8-ecea-4134-966a-d52dfec730e1 service placement] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:00.875982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ba9d-183d-4c71-ab6b-353a9eb163f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:00.876615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ba9d-183d-4c71-ab6b-353a9eb163f7 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:00.876894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ba9d-183d-4c71-ab6b-353a9eb163f7 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:00.877166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ba9d-183d-4c71-ab6b-353a9eb163f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:00.886066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1ba9d-183d-4c71-ab6b-353a9eb163f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:00.887648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1ba9d-183d-4c71-ab6b-353a9eb163f7 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:00.895892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1148/2285] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:38:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:38:01.628763 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:01.630577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:01.707847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb09b-fafe-4bea-8a66-39f984ca613d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:01.713623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:01.714255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:01.714541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:01.714807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] PATH_INFO: `/v3/users/7200fea87a8f411980e8280d654e3f7f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:01.715964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7200fea87a8f411980e8280d654e3f7f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:01.717936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:01.942840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7200fea87a8f411980e8280d654e3f7f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:02.012507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7200fea87a8f411980e8280d654e3f7f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:02.013268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7200fea87a8f411980e8280d654e3f7f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:02.031074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7200fea87a8f411980e8280d654e3f7f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:02.036955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:02.039354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7200fea87a8f411980e8280d654e3f7f'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:02.046579 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1138/2286] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:38:01 2022] DELETE /identity/v3/users/7200fea87a8f411980e8280d654e3f7f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:38:02.051326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:02.053393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:02.053393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:02.083423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:02.084701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:02.145049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:02.162781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1149/2287] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:02.173230 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:02.174655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:02.238778 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ecc63-ffe6-4d88-aeee-1548e6466c4f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:02.243818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75105549-e01a-4827-a39b-9d6bdaadd513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:02.244455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75105549-e01a-4827-a39b-9d6bdaadd513 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:02.244734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75105549-e01a-4827-a39b-9d6bdaadd513 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:02.244998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75105549-e01a-4827-a39b-9d6bdaadd513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:02.254339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75105549-e01a-4827-a39b-9d6bdaadd513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:02.255750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75105549-e01a-4827-a39b-9d6bdaadd513 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:02.265336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1139/2288] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:38:02.798083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:02.799561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:02.811686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a60298-fb04-48d1-9483-978d4deba610 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:02.816744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:02.817985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:02.818472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:02.818748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] PATH_INFO: `/v3/projects/9ce59252a5704781848f9f54640e674e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:02.820001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce59252a5704781848f9f54640e674e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:02.822764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:02.942838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce59252a5704781848f9f54640e674e', 'request_id': 'req-a61a4106-b58d-48e1-a3a8-3445b4e7a242'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:03.012371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce59252a5704781848f9f54640e674e', 'request_id': 'req-a61a4106-b58d-48e1-a3a8-3445b4e7a242'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:03.018724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce59252a5704781848f9f54640e674e', 'request_id': 'req-a61a4106-b58d-48e1-a3a8-3445b4e7a242'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:03.032018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] The token cache is being invalidate because project 9ce59252a5704781848f9f54640e674e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:38:03.032303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:03.032985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a61a4106-b58d-48e1-a3a8-3445b4e7a242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:03.034539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1150/2289] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:38:02 2022] DELETE /identity/v3/projects/9ce59252a5704781848f9f54640e674e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:38:03.040093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.040632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.040632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.060901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:03.062112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:03.132316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1140/2290] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:03.141224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.141759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.141759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.162620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:03.163864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:03.173067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1151/2291] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:03.177316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.188601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8310c6-9dbb-4d71-851b-6cf7695cb510 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.194903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.196547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.196850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.197126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.197819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:38:03.198139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.199852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.225482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1141/2292] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:38:03.229856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.242612 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446a400-d5a7-4b09-82b8-c0286b5c6b2c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.249305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.250067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.250412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.250784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.252555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.255175 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.366048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1152/2293] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:38:03 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:38:03.372306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.384227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691a8ae-737f-43c2-84a6-7e54c3827bcc admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.464577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.465572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.466041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.466506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.467580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.471183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.544959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1142/2294] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:38:03 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:38:03.549332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.562038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e36ca-7b43-4a71-9c39-addb95a860bf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.568433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.569176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.569525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.569803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.571083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.572793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.584747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1153/2295] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:38:03.588957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.600202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422c91f-6f3f-48a2-9892-f0e5a089222f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.605516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd49e0-3baf-47a1-8ae6-6c6e3dcf12ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.606131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd49e0-3baf-47a1-8ae6-6c6e3dcf12ef admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.606539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd49e0-3baf-47a1-8ae6-6c6e3dcf12ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.606813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd49e0-3baf-47a1-8ae6-6c6e3dcf12ef admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.607450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd49e0-3baf-47a1-8ae6-6c6e3dcf12ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.609081 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd49e0-3baf-47a1-8ae6-6c6e3dcf12ef admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.620992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1143/2296] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:38:03.625150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.640308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebb2c6-b079-4ba5-a129-480e46065e71 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.645248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.645966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.646240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.646507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] PATH_INFO: `/v3/projects/c6ef853624b14522b2bb2fca12bbb5c2/users/0703c9503c3f4be3bd857b17d7e69a86/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.665361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ef853624b14522b2bb2fca12bbb5c2, user_id=0703c9503c3f4be3bd857b17d7e69a86, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.667911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.734239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1154/2297] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:38:03 2022] PUT /identity/v3/projects/c6ef853624b14522b2bb2fca12bbb5c2/users/0703c9503c3f4be3bd857b17d7e69a86/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:38:03.747903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.748622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.749370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.764966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:03.791948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] MFA Rules not processed for user `0703c9503c3f4be3bd857b17d7e69a86`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:03.793656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:03.869385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1144/2298] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:38:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:03.880652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:03.882488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:03.949540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331609-b686-405c-a3fe-bb8684032dca admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:03.953439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699bce5-cf4f-4e6a-84ee-7ad6037e09f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:03.954047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699bce5-cf4f-4e6a-84ee-7ad6037e09f2 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:03.954325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699bce5-cf4f-4e6a-84ee-7ad6037e09f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:03.954594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699bce5-cf4f-4e6a-84ee-7ad6037e09f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:03.963945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699bce5-cf4f-4e6a-84ee-7ad6037e09f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:03.965421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699bce5-cf4f-4e6a-84ee-7ad6037e09f2 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:03.974527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1155/2299] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:38:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:38:03.999682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:04.069646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c99d-ead8-46c7-8db9-ea21ed1069ed None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:04.085013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:04.086307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:04.087662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:04.088521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] PATH_INFO: `/v3/users/0703c9503c3f4be3bd857b17d7e69a86` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:04.091061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0703c9503c3f4be3bd857b17d7e69a86)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:04.094054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:04.294011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0703c9503c3f4be3bd857b17d7e69a86'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:04.395214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0703c9503c3f4be3bd857b17d7e69a86'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:04.395850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0703c9503c3f4be3bd857b17d7e69a86'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:04.410767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0703c9503c3f4be3bd857b17d7e69a86'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:04.424898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0703c9503c3f4be3bd857b17d7e69a86'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:04.433348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1145/2300] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:38:03 2022] DELETE /identity/v3/users/0703c9503c3f4be3bd857b17d7e69a86 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:38:04.439091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:04.439927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:04.440090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:04.471513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:04.473278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:04.488075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:04.571620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1156/2301] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:04.581524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:04.583056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:04.598025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:04.644480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9f8c6-e4a0-45ac-a3a8-98ea34a3c6eb admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:04.649222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44412a4c-9bba-4112-9509-5bac22de7f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:04.649860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412a4c-9bba-4112-9509-5bac22de7f57 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:04.650140 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412a4c-9bba-4112-9509-5bac22de7f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:04.650413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412a4c-9bba-4112-9509-5bac22de7f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:04.659595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44412a4c-9bba-4112-9509-5bac22de7f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:04.661156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44412a4c-9bba-4112-9509-5bac22de7f57 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:04.671646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1146/2302] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:38:05.312402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:05.314384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:05.337292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed7ccb-5bf4-4266-83c2-a98e24ae2438 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:05.337292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:05.337292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:05.337292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:05.338141 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] PATH_INFO: `/v3/projects/c6ef853624b14522b2bb2fca12bbb5c2` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:05.339891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ef853624b14522b2bb2fca12bbb5c2)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:05.341863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:05.879715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ef853624b14522b2bb2fca12bbb5c2', 'request_id': 'req-502c4a3d-c516-4ca2-994f-7bba5f887bff'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:06.001489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ef853624b14522b2bb2fca12bbb5c2', 'request_id': 'req-502c4a3d-c516-4ca2-994f-7bba5f887bff'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:06.008592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ef853624b14522b2bb2fca12bbb5c2', 'request_id': 'req-502c4a3d-c516-4ca2-994f-7bba5f887bff'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:06.020556 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] The token cache is being invalidate because project c6ef853624b14522b2bb2fca12bbb5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:38:06.020850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:06.021536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:38:06.023319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1157/2303] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:38:05 2022] DELETE /identity/v3/projects/c6ef853624b14522b2bb2fca12bbb5c2 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:38:14.721045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=20 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:16.017212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:27.905734 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c657009-743e-4e10-a8cd-1d93760b3f0f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:27.906266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c657009-743e-4e10-a8cd-1d93760b3f0f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:27.906266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c657009-743e-4e10-a8cd-1d93760b3f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:27.926417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2c657009-743e-4e10-a8cd-1d93760b3f0f None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:27.927748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2c657009-743e-4e10-a8cd-1d93760b3f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:27.984444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1147/2304] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 00:38:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:27.997258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:27.998665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:28.071312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4a3d-c516-4ca2-994f-7bba5f887bff admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:28.076307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95345a53-0ffb-4735-a042-abba21019bf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:28.076936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345a53-0ffb-4735-a042-abba21019bf5 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:28.077247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345a53-0ffb-4735-a042-abba21019bf5 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:28.078015 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95345a53-0ffb-4735-a042-abba21019bf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:28.086506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95345a53-0ffb-4735-a042-abba21019bf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:28.088072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95345a53-0ffb-4735-a042-abba21019bf5 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:28.096047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1158/2305] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:38:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:38:37.977688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=25 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:38.092831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=32 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:39:02.341525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35195e41-3da0-4bb2-b1cb-b72757e2aff5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:02.350881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35195e41-3da0-4bb2-b1cb-b72757e2aff5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:02.350881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35195e41-3da0-4bb2-b1cb-b72757e2aff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:02.361998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-35195e41-3da0-4bb2-b1cb-b72757e2aff5 None None] MFA Rules not processed for user `45422a67a78f4784abc0f18ad84778f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:39:02.363186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-35195e41-3da0-4bb2-b1cb-b72757e2aff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:02.419948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1148/2306] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:39:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:39:04.682818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:04.683373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:04.683373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:04.703409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] MFA Rules not processed for user `77d23c99308c414f939761b07d329e03`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:39:04.704649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:04.760297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1159/2307] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:39:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:39:12.413016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=25 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:39:14.752867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=25 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:39:23.375169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:23.375740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:23.376038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:23.398703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] MFA Rules not processed for user `2a927401086b4a649919a6071b4a0fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:39:23.400953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:23.463851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1149/2308] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:39:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:39:33.456738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=25 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:39:48.135571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:39:48.137703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:48.197051 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d770cbd-a713-4184-8975-1a00361df989 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:39:48.200807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:39:48.201408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:48.201711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:48.201982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:48.203695 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:48.275314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:39:48.276776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd11068-88a8-4f80-8182-6f5e889c97a6 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:39:48.289146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1160/2309] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:39:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:39:49.416280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:39:49.417902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:49.485112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f121-5fe2-4aae-ba33-aaa88dffb5f0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:39:49.488001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:39:49.488658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:49.488939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:49.489240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:49.491670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:49.515360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:39:49.515360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:39:49.529603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1150/2310] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:39:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:39:52.942235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:52.942818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:52.942818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:52.962835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] MFA Rules not processed for user `022c4688d9ce4df9a99ae5fa84e1d871`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:39:52.964045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:53.019719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1161/2311] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:39:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:39:59.523767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=26 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:01.239449 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:01.243113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:01.258203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ad03-bd77-46ba-bfef-bacda1b9e908 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:01.263050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:01.263659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:01.263939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:01.264899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:01.266748 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:01.279948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:01.281534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:01.292156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1151/2312] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:03.012756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=79 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:08.416196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:08.426095 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd6a6-5ca9-4892-85d6-4687e290fddb None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:08.429814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:08.430592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:08.430866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:08.431140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:08.433073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:08.444824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:08.446790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:08.463195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1162/2313] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:11.286953 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:14.990488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:15.001098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cc012-9d4d-4115-816a-c6b8cc891c53 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:15.005316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:15.005943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:15.006246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:15.006517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:15.008393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:15.018909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:15.020414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:15.029236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1152/2314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:40:18.455191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=6 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:21.033780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:21.045005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c3ae-048d-4537-82e2-4efb857f1dec service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:21.049894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:21.050770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:21.051286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:21.051959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:21.054569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:21.143437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:21.145729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:21.158712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1163/2315] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:22.579330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:22.580841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:22.709442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c187d-b902-45cc-9124-51c5085f1166 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:22.717737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:22.718561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:22.718702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:22.718969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:22.720848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:22.735816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:22.737716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:22.749808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1153/2316] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:24.628021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:24.630691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:24.648466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea741142-1fde-4d59-9381-ddc0371cd138 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:24.655243 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:24.656472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:24.656731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:24.657191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:24.659232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:24.672588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:24.674330 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:24.687739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1164/2317] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:25.860507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:25.862990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:25.878153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5811f-ee67-43a8-bf63-dc516a1b32a9 service placement] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:25.882892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:25.883554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:25.883832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:25.884101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] PATH_INFO: `/v3/users/77d23c99308c414f939761b07d329e03` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:25.885251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77d23c99308c414f939761b07d329e03)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:25.887874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:26.063203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77d23c99308c414f939761b07d329e03'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.134580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77d23c99308c414f939761b07d329e03'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.135393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77d23c99308c414f939761b07d329e03'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.150443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77d23c99308c414f939761b07d329e03'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.159489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-e373ebcc-4b8c-40ba-b42b-ce35052536c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77d23c99308c414f939761b07d329e03'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.170010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1154/2318] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:40:25 2022] DELETE /identity/v3/users/77d23c99308c414f939761b07d329e03 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:26.491785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:26.511323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112053f4-75aa-4a90-9843-35f35fe9e7c4 service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:26.576795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:26.577476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:26.577924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:26.578204 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] PATH_INFO: `/v3/projects/d8aa6e0e8e6b492f8e74eea0e830c0bc` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:26.579377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8aa6e0e8e6b492f8e74eea0e830c0bc)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:26.581673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:26.726285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8aa6e0e8e6b492f8e74eea0e830c0bc', 'request_id': 'req-1eaf43b6-69fe-4911-b335-2844e913ba08'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.826172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8aa6e0e8e6b492f8e74eea0e830c0bc', 'request_id': 'req-1eaf43b6-69fe-4911-b335-2844e913ba08'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.833425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8aa6e0e8e6b492f8e74eea0e830c0bc', 'request_id': 'req-1eaf43b6-69fe-4911-b335-2844e913ba08'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.845912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] The token cache is being invalidate because project d8aa6e0e8e6b492f8e74eea0e830c0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:40:26.846200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.846967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1eaf43b6-69fe-4911-b335-2844e913ba08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:26.848492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1165/2319] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:40:26 2022] DELETE /identity/v3/projects/d8aa6e0e8e6b492f8e74eea0e830c0bc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:26.853511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:26.853792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:26.854062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:26.874248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:26.875467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:26.921932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:26.937662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1155/2320] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:26.945399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:26.945683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:26.945959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:26.967951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:26.969247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:26.977285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1166/2321] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:26.981756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:26.992489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3db22-ad8d-41a6-b679-64cfdb4a0943 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:26.997320 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:26.997935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:26.998218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:26.998487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:26.999144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:40:26.999452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.001110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.014238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1156/2322] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:40:27.018913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:27.024961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:27.030348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fb491-7f3e-42aa-9734-c58ae0d6604f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:27.035505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:27.036380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.036651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.036917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.037559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.039535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.159442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1167/2323] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:40:27 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:40:27.163934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:27.174837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce2dd5-ccf6-4126-b0aa-e42d9bd04abf admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:27.244305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:27.244964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.245244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.245544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.246251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.249058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.333693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1157/2324] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:40:27 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:40:27.334938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:27.349210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df724f6-8ba8-472a-92e2-b2bcbd4ecf12 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:27.355224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:27.355908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.356193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.356500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.357254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.358907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.376793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1168/2325] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:40:27.377469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:27.393553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532d7d6-b8ce-406d-833c-c447d2e31e28 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:27.399902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faab44-685c-4023-87b2-befcb6753736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:27.400771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faab44-685c-4023-87b2-befcb6753736 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.401127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faab44-685c-4023-87b2-befcb6753736 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.401529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faab44-685c-4023-87b2-befcb6753736 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.402300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faab44-685c-4023-87b2-befcb6753736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.404743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6faab44-685c-4023-87b2-befcb6753736 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.421768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1158/2326] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:40:27.428159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:27.445636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5510a-d99f-4c29-9785-c9ada4c60ada admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:27.453692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:27.454758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.455242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.455724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] PATH_INFO: `/v3/projects/76f177d4bb574dfa872bef4fabad6284/users/7fb8b52f977f43e19d08ac06ed98f531/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.482472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f177d4bb574dfa872bef4fabad6284, user_id=7fb8b52f977f43e19d08ac06ed98f531, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.482707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.579672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1169/2327] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:40:27 2022] PUT /identity/v3/projects/76f177d4bb574dfa872bef4fabad6284/users/7fb8b52f977f43e19d08ac06ed98f531/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:27.581219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.581509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.581779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.635270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] MFA Rules not processed for user `7fb8b52f977f43e19d08ac06ed98f531`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:27.636887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:27.737842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1159/2328] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:27.750779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:27.753682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:27.832999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aebcc9-4110-4c17-b57a-14a073b96385 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:27.839658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdedbf6-67b9-448f-9203-d2689a656d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:27.841220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdedbf6-67b9-448f-9203-d2689a656d19 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:27.841890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdedbf6-67b9-448f-9203-d2689a656d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:27.842166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdedbf6-67b9-448f-9203-d2689a656d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:27.855786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdedbf6-67b9-448f-9203-d2689a656d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:27.857777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdedbf6-67b9-448f-9203-d2689a656d19 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:27.868881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1170/2329] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:28.720468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:28.723784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:28.838762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:28.907684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965b107-2108-4547-b719-51e54a4f0eef None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:28.920989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:28.928414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:28.928414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:28.928414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:28.928904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:28.946601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:28.948750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ef772-b4ec-4821-aba9-83a8c08b7ce7 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:28.967710 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1160/2330] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:34.807835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:34.808274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:34.808274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:34.845218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] MFA Rules not processed for user `ac2476465c4c4bd2b1cf81934b6e9a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:34.847012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:34.958047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1171/2331] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:35.339098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:35.339661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:35.341138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:35.362401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] MFA Rules not processed for user `45422a67a78f4784abc0f18ad84778f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:35.363683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:35.366370 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:35.369985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:35.438081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179fa79-248d-4072-a5e3-c533d8c29a9e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:35.442269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:35.442922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:35.443196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:35.443463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:35.445197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:35.451472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1161/2332] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:35.459922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:35.470668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:35.470668 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:35.471282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1172/2333] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:40:35.781113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:35.782931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:35.798284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73c8d0-1e21-41d1-8a23-21d5462a884a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:35.803339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:35.803577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:35.803855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:35.804122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:35.805989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:35.816420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:35.817860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a891369-946e-466a-b4b0-6b4f4ee4e593 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:35.827999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1162/2334] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:38.864073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:38.930734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151f56-46f3-4490-8471-e519c4589f29 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:38.936259 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:38.936884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:38.937099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:38.937367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] PATH_INFO: `/v3/users/7fb8b52f977f43e19d08ac06ed98f531` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:38.939083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb8b52f977f43e19d08ac06ed98f531)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:38.940643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:39.145989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb8b52f977f43e19d08ac06ed98f531'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:39.160906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb8b52f977f43e19d08ac06ed98f531'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:39.169407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb8b52f977f43e19d08ac06ed98f531'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:39.242824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb8b52f977f43e19d08ac06ed98f531'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:39.243516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb8b52f977f43e19d08ac06ed98f531'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:39.250521 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1173/2335] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:40:38 2022] DELETE /identity/v3/users/7fb8b52f977f43e19d08ac06ed98f531 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:39.255194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:39.255662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:39.255758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:39.277343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:39.278623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:39.340513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1163/2336] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:39.350999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:39.352454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:39.417978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f8fc-11d0-4365-8bfb-7f7c39c2e7d5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:39.422551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd788d5-2927-495c-86c4-36fff1e27089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:39.423172 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd788d5-2927-495c-86c4-36fff1e27089 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:39.423449 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd788d5-2927-495c-86c4-36fff1e27089 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:39.423719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd788d5-2927-495c-86c4-36fff1e27089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:39.433877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd788d5-2927-495c-86c4-36fff1e27089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:39.435663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd788d5-2927-495c-86c4-36fff1e27089 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:39.447699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1174/2337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:40.118090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:40.119685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:40.158317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9054849-b6e5-4fa6-9451-23b480d3fe49 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:40.168967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:40.171074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:40.171630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:40.173407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] PATH_INFO: `/v3/projects/76f177d4bb574dfa872bef4fabad6284` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:40.176082 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f177d4bb574dfa872bef4fabad6284)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:40.197203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:40.416368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f177d4bb574dfa872bef4fabad6284', 'request_id': 'req-9d327735-199e-41bd-8441-7833e3ef8f11'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:40.421794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:40.425407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f177d4bb574dfa872bef4fabad6284', 'request_id': 'req-9d327735-199e-41bd-8441-7833e3ef8f11'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:40.440130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f177d4bb574dfa872bef4fabad6284', 'request_id': 'req-9d327735-199e-41bd-8441-7833e3ef8f11'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:40.540056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] The token cache is being invalidate because project 76f177d4bb574dfa872bef4fabad6284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:40:40.540369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:40.541113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9d327735-199e-41bd-8441-7833e3ef8f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:40.542702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1164/2338] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:40:40 2022] DELETE /identity/v3/projects/76f177d4bb574dfa872bef4fabad6284 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:40.548610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:40.549127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:40.549221 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:40.570826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:40.572162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:40.590824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:40.661575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1175/2339] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:40.670382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25955287-942c-472b-a4f3-5709349c8097 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:40.670698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25955287-942c-472b-a4f3-5709349c8097 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:40.670949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25955287-942c-472b-a4f3-5709349c8097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:40.694324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-25955287-942c-472b-a4f3-5709349c8097 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:40.695619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-25955287-942c-472b-a4f3-5709349c8097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:40.733125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1165/2340] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:40.734098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:40.754919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6c4b6-7ff2-4023-b4f1-88c65454bd0a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:40.763611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:40.764559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:40.764879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:40.765152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:40.765983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:40:40.766267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:40.768164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:40.790102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1176/2341] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:40:40.796019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25955287-942c-472b-a4f3-5709349c8097 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:40.807039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25955287-942c-472b-a4f3-5709349c8097 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:40.812931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:40.813917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:40.814200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:40.814502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:40.815165 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:40.817321 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:40.973431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1166/2342] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:40:40 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:40:40.973939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:40.988956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd10d95-ca13-4b71-8831-515b45174c53 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:41.120582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:41.121439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:41.121733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:41.122133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:41.123599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:41.125761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:41.243734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1177/2343] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:40:40 2022] POST /identity/v3/users => generated 291 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:40:41.245814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:41.282963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf028fd3-8ffb-4ba7-8050-ad695a68ab4a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:41.295114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:41.295899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:41.296729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:41.296729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:41.297458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:41.299542 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:41.316845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1167/2344] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:40:41.317654 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:41.351569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029300f3-d620-4a18-8a76-769dddd4778c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:41.369233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20337-013c-40aa-8134-b9df92d9d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:41.370027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20337-013c-40aa-8134-b9df92d9d149 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:41.370413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20337-013c-40aa-8134-b9df92d9d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:41.370919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20337-013c-40aa-8134-b9df92d9d149 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:41.371774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20337-013c-40aa-8134-b9df92d9d149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:41.374113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20337-013c-40aa-8134-b9df92d9d149 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:41.388974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1178/2345] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:40:41.393495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:41.412557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a253e-fc78-468e-a5ec-34df3b6dbdf5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:41.425102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:41.426540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:41.427359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:41.427844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] PATH_INFO: `/v3/projects/b57b88b898374a37b0851be8ac16613f/users/3026dc987556414b919934cae52104d7/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:41.454779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57b88b898374a37b0851be8ac16613f, user_id=3026dc987556414b919934cae52104d7, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:41.460192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:41.573519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1168/2346] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:40:41 2022] PUT /identity/v3/projects/b57b88b898374a37b0851be8ac16613f/users/3026dc987556414b919934cae52104d7/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:41.585008 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:41.586052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:41.586166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:41.632867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] MFA Rules not processed for user `3026dc987556414b919934cae52104d7`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:41.634128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:41.722743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1179/2347] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:41.728117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:41.809326 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5d3a2-5aad-48a8-89a7-e755fba2163a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:41.816871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:41.817701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:41.818080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:41.818584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:41.819338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:41.821475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:41.936445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1169/2348] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:40:41 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:40:41.941152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:41.957541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb900fdb-a2d4-42f0-8edf-3318bb12aad2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:41.977607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:42.047088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.048002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.048472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.048916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.049762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.051952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.152278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1180/2349] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:40:41 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:40:42.156938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:42.180291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec329a-a22e-441d-9f3f-8463a93ed686 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:42.190233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.207487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.207777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.208311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.208947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.222341 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.249159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1170/2350] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:40:42.265134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:42.265459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:42.267632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:42.281311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd25c-af3d-4b3d-af27-ddf8310bd13f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:42.295856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.297726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.297726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.297726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] PATH_INFO: `/v3/projects/f042e2ba6b2f4c588acd39387cc3301f/users/8facb8427e114043aaba2bbbb705fd0c/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.340280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f042e2ba6b2f4c588acd39387cc3301f, user_id=8facb8427e114043aaba2bbbb705fd0c, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.344742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.346619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:42.377685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3bce-0cd7-4e9f-8f66-a94553cfb502 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:42.386664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.387700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.388375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.388932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.391513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:42.407501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.409638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.430569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1171/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:42.478324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1181/2352] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:40:42 2022] PUT /identity/v3/projects/f042e2ba6b2f4c588acd39387cc3301f/users/8facb8427e114043aaba2bbbb705fd0c/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:42.484584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:42.500544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce146f-f16c-41e0-a961-b979489d743f service nova] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:42.599755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c68b08-c52c-4912-a7d8-1c4e0f52eb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.600885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68b08-c52c-4912-a7d8-1c4e0f52eb1d admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.601378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68b08-c52c-4912-a7d8-1c4e0f52eb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.601751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68b08-c52c-4912-a7d8-1c4e0f52eb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.602512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c68b08-c52c-4912-a7d8-1c4e0f52eb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.604572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c68b08-c52c-4912-a7d8-1c4e0f52eb1d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.619408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1172/2353] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:40:42.623844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:42.647147 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d8295-70ab-4800-860b-b3aa6975c9e1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:42.655636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.659692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.659692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.659692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] PATH_INFO: `/v3/projects/f042e2ba6b2f4c588acd39387cc3301f/users/8facb8427e114043aaba2bbbb705fd0c/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.659692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f042e2ba6b2f4c588acd39387cc3301f, user_id=8facb8427e114043aaba2bbbb705fd0c, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.661731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.673002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1182/2354] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:40:42 2022] PUT /identity/v3/projects/f042e2ba6b2f4c588acd39387cc3301f/users/8facb8427e114043aaba2bbbb705fd0c/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:42.685403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.685782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.686060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.725197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] MFA Rules not processed for user `8facb8427e114043aaba2bbbb705fd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:42.727164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:42.817615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1173/2355] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:42.831011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:42.837794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:42.922025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab8e30-e4cf-4494-b3b2-d7aa4dfce7be admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:42.926778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ca4f-14ca-4386-9985-4351a7d21881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:42.928036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ca4f-14ca-4386-9985-4351a7d21881 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:42.928558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ca4f-14ca-4386-9985-4351a7d21881 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:42.930112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ca4f-14ca-4386-9985-4351a7d21881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:42.944319 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178ca4f-14ca-4386-9985-4351a7d21881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:42.946181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178ca4f-14ca-4386-9985-4351a7d21881 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:42.975252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1183/2356] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:40:43.151950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:43.164670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7269a-da04-49a1-ad07-571799b20b98 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:43.168720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:43.169510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:43.169859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:43.170231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:43.253173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:43.255029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:43.261059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:43.266828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1174/2357] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:51.041681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:51.042263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:51.042381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:51.062819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:51.064053 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:51.125124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1184/2358] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:40:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:51.289617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:51.291286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:51.383481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aca1d5-c039-4207-90d2-5db2dcf058d4 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:51.393658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c028858-c164-4f08-8e13-a5884377a6d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:51.396942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c028858-c164-4f08-8e13-a5884377a6d2 service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:51.396942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c028858-c164-4f08-8e13-a5884377a6d2 service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:51.396942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c028858-c164-4f08-8e13-a5884377a6d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:51.405610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c028858-c164-4f08-8e13-a5884377a6d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:51.407399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c028858-c164-4f08-8e13-a5884377a6d2 service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:51.418018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1175/2359] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:40:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:40:56.528292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:56.555436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:56.635839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796171b-a5b0-49ca-a995-543c5abf9aa2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:56.641795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:56.642667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:56.643098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:56.643543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] PATH_INFO: `/v3/users/3026dc987556414b919934cae52104d7` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:56.644893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3026dc987556414b919934cae52104d7)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:56.646980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:56.868206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3026dc987556414b919934cae52104d7'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:56.891161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3026dc987556414b919934cae52104d7'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:56.900258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3026dc987556414b919934cae52104d7'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:56.999586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3026dc987556414b919934cae52104d7'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:57.000545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3026dc987556414b919934cae52104d7'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:57.010306 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1185/2360] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:40:56 2022] DELETE /identity/v3/users/3026dc987556414b919934cae52104d7 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:57.015452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:57.015998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:57.015998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:57.058852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:40:57.061869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:57.197227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1176/2361] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:40:57.204578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:57.206226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:57.367460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e29d7b-5287-47de-9fa7-51e8f977e7ed admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:57.383193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:57.383836 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:57.384116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:57.384430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:57.409190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:57.410932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:57.436558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1186/2362] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:58.322459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:58.325134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:40:58.352854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674d0a1-d70a-44fd-82f2-d07207a06f8f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:58.371901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:58.373083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:58.373427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:58.373918 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] PATH_INFO: `/v3/projects/b57b88b898374a37b0851be8ac16613f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:58.375471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57b88b898374a37b0851be8ac16613f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:58.377596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:58.531557 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57b88b898374a37b0851be8ac16613f', 'request_id': 'req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:58.538630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57b88b898374a37b0851be8ac16613f', 'request_id': 'req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:58.546469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57b88b898374a37b0851be8ac16613f', 'request_id': 'req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:58.678901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] The token cache is being invalidate because project b57b88b898374a37b0851be8ac16613f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:40:58.679749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:58.681150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:58.683072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1177/2363] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:40:58 2022] DELETE /identity/v3/projects/b57b88b898374a37b0851be8ac16613f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:40:58.688027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:58.699353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a99346-dc08-40d8-b5fe-64908d1210c3 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:58.747761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:58.785142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:58.786030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:58.786390 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:58.786796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] PATH_INFO: `/v3/users/8facb8427e114043aaba2bbbb705fd0c` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:58.789010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8facb8427e114043aaba2bbbb705fd0c)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:58.794260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:59.051110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8facb8427e114043aaba2bbbb705fd0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:59.069364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8facb8427e114043aaba2bbbb705fd0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:59.078569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8facb8427e114043aaba2bbbb705fd0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:59.163214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8facb8427e114043aaba2bbbb705fd0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:59.164937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-822c64b0-22a4-4e81-8f57-ac2857ca41d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8facb8427e114043aaba2bbbb705fd0c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:40:59.176604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1187/2364] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:40:58 2022] DELETE /identity/v3/users/8facb8427e114043aaba2bbbb705fd0c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:00.375773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:00.377350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:00.399835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4cd1-a967-4519-8e77-a58f97bf4cef admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:00.422080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:00.528194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:00.528972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:00.529423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:00.530214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] PATH_INFO: `/v3/projects/f042e2ba6b2f4c588acd39387cc3301f` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:00.533200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f042e2ba6b2f4c588acd39387cc3301f)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:00.535162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:00.725964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f042e2ba6b2f4c588acd39387cc3301f', 'request_id': 'req-94283787-0158-4259-aebc-97844b292ccd'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:00.740961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f042e2ba6b2f4c588acd39387cc3301f', 'request_id': 'req-94283787-0158-4259-aebc-97844b292ccd'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:00.757645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f042e2ba6b2f4c588acd39387cc3301f', 'request_id': 'req-94283787-0158-4259-aebc-97844b292ccd'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:00.875936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] The token cache is being invalidate because project f042e2ba6b2f4c588acd39387cc3301f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:00.880722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:00.880722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-94283787-0158-4259-aebc-97844b292ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:00.880722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1178/2365] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:00 2022] DELETE /identity/v3/projects/f042e2ba6b2f4c588acd39387cc3301f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:00.884117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67332469-d576-4d06-82b3-bb575d094857 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:00.884617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67332469-d576-4d06-82b3-bb575d094857 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:00.884748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67332469-d576-4d06-82b3-bb575d094857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:00.911917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-67332469-d576-4d06-82b3-bb575d094857 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:00.913512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-67332469-d576-4d06-82b3-bb575d094857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:01.020185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1188/2366] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:01.027703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.027993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.028270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.069276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:01.075097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:01.088919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1179/2367] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:01.093379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67332469-d576-4d06-82b3-bb575d094857 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.107646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67332469-d576-4d06-82b3-bb575d094857 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.122180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.122816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.123161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.123436 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.124099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:41:01.124424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.126029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.151752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1189/2368] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:41:01.157143 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.167306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a72f5-d091-4819-9247-cd1203884d98 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.174935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.175715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.176269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.176680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.177492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.180576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.307302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1180/2369] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:01 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:41:01.311468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.331589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3d771-3c35-496b-9571-807b035b28e5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.398517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:01.423156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.423964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.424494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.424865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.431896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.433724 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.534414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1190/2370] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:01 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:01.537312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.553418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fca85-9786-4d22-9525-820049cc687f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.563781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.566269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:01.569054 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.569936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.573518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.573518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.573518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.573518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.587462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1181/2371] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:01.593505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.610120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67525065-ce9c-427e-b526-6ced2b16b796 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.618450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.619933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.620485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.620968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.622640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.625434 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.648990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1182/2372] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:01.651092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:01.666002 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca570e-77fa-46e1-9792-12a20b514325 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.674747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.675743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.676146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.676558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] PATH_INFO: `/v3/projects/26f3ac19e3ba4ac4a7e0efab257a294e/users/93c7e628aad54b7caf2293799c13f8fa/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.684925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14d69c-5b64-4797-81b9-982677bc2b8d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:01.697682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f3ac19e3ba4ac4a7e0efab257a294e, user_id=93c7e628aad54b7caf2293799c13f8fa, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.699716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:01.701045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.702263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.702404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.702830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.706570 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:01.728482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:01.731475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a836-4131-451d-ad28-e72e96de0216 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:01.753748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1191/2373] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:01.803580 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1183/2374] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:01 2022] PUT /identity/v3/projects/26f3ac19e3ba4ac4a7e0efab257a294e/users/93c7e628aad54b7caf2293799c13f8fa/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:01.805649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:01.805936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:01.806288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:01.855344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] MFA Rules not processed for user `93c7e628aad54b7caf2293799c13f8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:01.857384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:01.989637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1192/2375] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:41:01.993803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:02.056468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:02.110942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a63e-d029-47c9-9e1b-c36b286a899a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:02.116521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:02.117312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:02.117753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:02.118364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:02.119181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:02.121235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:02.238446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1184/2376] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:01 2022] POST /identity/v3/projects => generated 369 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:02.239759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:02.269912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eac394-e282-4d9e-bdf0-d51f1c05dba7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:02.353033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:02.372694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:02.373788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:02.374460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:02.375219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:02.376623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:02.379431 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:02.809275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1193/2377] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:02 2022] POST /identity/v3/users => generated 291 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:02.813847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:02.824998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027949-4aad-469d-aafe-cc4e03356345 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:02.835747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:02.837010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:02.837336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:02.837795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:02.838463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:02.840161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:02.853637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1185/2378] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:02.858905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:02.869909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3926d-857d-4f5f-8e7e-2eccc55b2c21 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:02.875305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:02.876062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:02.876361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:02.876689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] PATH_INFO: `/v3/projects/0bd8b5d9c8f24985ad743e74162f6324/users/c29ed6e565c444ee9a2acb5f436a3840/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:02.901198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd8b5d9c8f24985ad743e74162f6324, user_id=c29ed6e565c444ee9a2acb5f436a3840, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:02.904065 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:02.987265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1194/2379] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:02 2022] PUT /identity/v3/projects/0bd8b5d9c8f24985ad743e74162f6324/users/c29ed6e565c444ee9a2acb5f436a3840/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:02.991971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:03.009392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a92cc-897d-492c-841c-e62676f1976e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:03.122751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d145aa-4d67-4b6e-ae7f-df47d0f868ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:03.123926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d145aa-4d67-4b6e-ae7f-df47d0f868ee admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:03.124205 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d145aa-4d67-4b6e-ae7f-df47d0f868ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:03.124507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d145aa-4d67-4b6e-ae7f-df47d0f868ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:03.125160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d145aa-4d67-4b6e-ae7f-df47d0f868ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:03.126902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d145aa-4d67-4b6e-ae7f-df47d0f868ee admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:03.149633 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1186/2380] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:03.154849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:03.166778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da8ff6-6a36-415e-8eea-d15dd742caa9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:03.172662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:03.173625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:03.178619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:03.178619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] PATH_INFO: `/v3/projects/0bd8b5d9c8f24985ad743e74162f6324/users/c29ed6e565c444ee9a2acb5f436a3840/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:03.179401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd8b5d9c8f24985ad743e74162f6324, user_id=c29ed6e565c444ee9a2acb5f436a3840, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:03.182212 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:03.197901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1195/2381] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:03 2022] PUT /identity/v3/projects/0bd8b5d9c8f24985ad743e74162f6324/users/c29ed6e565c444ee9a2acb5f436a3840/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:03.210100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:03.210393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:03.210682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:03.262780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] MFA Rules not processed for user `c29ed6e565c444ee9a2acb5f436a3840`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:03.264306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:03.360880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1187/2382] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:03.371857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:03.376109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:03.444810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63dc8-ade5-46b9-8a17-4cb287a0d38a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:03.449378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae15c0-24f4-4c49-a7ea-c69eeb9029dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:03.450090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae15c0-24f4-4c49-a7ea-c69eeb9029dd service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:03.450469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae15c0-24f4-4c49-a7ea-c69eeb9029dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:03.450746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae15c0-24f4-4c49-a7ea-c69eeb9029dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:03.466321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae15c0-24f4-4c49-a7ea-c69eeb9029dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:03.468593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae15c0-24f4-4c49-a7ea-c69eeb9029dd service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:03.483916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1196/2383] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:06.494436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:06.521949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:06.557934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a808338-a966-4037-ac39-1b7ff369ed2a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:06.563212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:06.563834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:06.564110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:06.564417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] PATH_INFO: `/v3/users/93c7e628aad54b7caf2293799c13f8fa` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:06.565543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c7e628aad54b7caf2293799c13f8fa)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:06.567461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:06.751175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c7e628aad54b7caf2293799c13f8fa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:06.814200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c7e628aad54b7caf2293799c13f8fa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:06.814819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c7e628aad54b7caf2293799c13f8fa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:06.828033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c7e628aad54b7caf2293799c13f8fa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:06.835115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c7e628aad54b7caf2293799c13f8fa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:06.841005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1188/2384] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:06 2022] DELETE /identity/v3/users/93c7e628aad54b7caf2293799c13f8fa => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:06.846027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:06.846525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:06.846659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:06.866395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:06.867611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:06.927465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1197/2385] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:06.937050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:06.938436 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:06.995373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749a151-9251-4e44-8189-319befb53b03 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:06.999027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:06.999797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:07.000074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:07.000364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:07.009466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:07.010905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:07.020694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1189/2386] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:07.430566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:07.432230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:07.445632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d871b8-6170-4fe7-ac0a-f5b3f373ded9 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:07.450783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:07.451426 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:07.451702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:07.452651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] PATH_INFO: `/v3/projects/26f3ac19e3ba4ac4a7e0efab257a294e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:07.453885 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f3ac19e3ba4ac4a7e0efab257a294e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:07.455847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:07.557596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:07.565294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f3ac19e3ba4ac4a7e0efab257a294e', 'request_id': 'req-ebfd5079-040a-469d-8350-8261be8f3c34'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:07.661229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f3ac19e3ba4ac4a7e0efab257a294e', 'request_id': 'req-ebfd5079-040a-469d-8350-8261be8f3c34'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:07.668500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f3ac19e3ba4ac4a7e0efab257a294e', 'request_id': 'req-ebfd5079-040a-469d-8350-8261be8f3c34'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:07.680492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] The token cache is being invalidate because project 26f3ac19e3ba4ac4a7e0efab257a294e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:07.680788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:07.681507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:07.683023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1198/2387] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:07 2022] DELETE /identity/v3/projects/26f3ac19e3ba4ac4a7e0efab257a294e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:07.687687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:07.698251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af294c9-4d1b-46ba-9870-40e6f12f1003 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:07.744038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:07.756954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:07.757587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:07.757873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:07.758149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] PATH_INFO: `/v3/users/c29ed6e565c444ee9a2acb5f436a3840` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:07.759282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29ed6e565c444ee9a2acb5f436a3840)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:07.761333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:07.962887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29ed6e565c444ee9a2acb5f436a3840'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.057549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29ed6e565c444ee9a2acb5f436a3840'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.058394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29ed6e565c444ee9a2acb5f436a3840'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.072830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29ed6e565c444ee9a2acb5f436a3840'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.080519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6ef0735f-8846-429f-9cbc-185318d0b5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29ed6e565c444ee9a2acb5f436a3840'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.090315 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1190/2388] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:07 2022] DELETE /identity/v3/users/c29ed6e565c444ee9a2acb5f436a3840 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:08.535835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:08.537538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:08.552828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd5079-040a-469d-8350-8261be8f3c34 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:08.615842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:08.616617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:08.616887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:08.617164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] PATH_INFO: `/v3/projects/0bd8b5d9c8f24985ad743e74162f6324` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:08.621013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd8b5d9c8f24985ad743e74162f6324)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:08.621013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:08.759057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd8b5d9c8f24985ad743e74162f6324', 'request_id': 'req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.826354 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd8b5d9c8f24985ad743e74162f6324', 'request_id': 'req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.835845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd8b5d9c8f24985ad743e74162f6324', 'request_id': 'req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.845821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] The token cache is being invalidate because project 0bd8b5d9c8f24985ad743e74162f6324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:08.846113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.846789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7c6eb274-b6e6-4ca7-94ed-69317bb9b6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:08.848409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1199/2389] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:08 2022] DELETE /identity/v3/projects/0bd8b5d9c8f24985ad743e74162f6324 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:08.853583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:08.854085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:08.854085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:08.883219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:08.884556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:08.946745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1191/2390] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:08.954992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:08.955277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:08.955554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:08.977129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:08.978386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:08.987272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1200/2391] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:08.991525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.002215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39c2ac-adce-4e44-8d55-ed242f46bc08 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.007181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.007805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.008073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.008369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.009199 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:41:09.009411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.011604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.028099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1192/2392] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:41:09.035324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.045288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37ae6c-e322-499d-a5df-6637a857625f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.050568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.051168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.051451 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.053887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.053887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.054309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.125067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1201/2393] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:09 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:09.129284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.139952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839a368-f5ca-4b35-9945-69a09653f15b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.192108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:09.201549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.202715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.202715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.202922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.203364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.205382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.312594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1193/2394] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:09 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:09.316582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.327949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5f07-9670-42e3-874e-4477faa07f05 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.332856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.333493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.333772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.334073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.334744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.336371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.347313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1202/2395] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:09.351368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.362832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666ce42-8877-4728-8b00-dc6754aa3825 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.367694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db6a0d-236b-459f-aacb-b819bccb647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.369200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6a0d-236b-459f-aacb-b819bccb647a admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.369200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6a0d-236b-459f-aacb-b819bccb647a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.369200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6a0d-236b-459f-aacb-b819bccb647a admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.369482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db6a0d-236b-459f-aacb-b819bccb647a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.371118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db6a0d-236b-459f-aacb-b819bccb647a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.383203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1194/2396] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:09.387513 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.400194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd041756-f05f-4d79-be85-d8078e249bf9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.407733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.408526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.408830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.409089 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] PATH_INFO: `/v3/projects/f4e1d34f03ca4c8a85eedb63a57311f2/users/7da7a5b617164ff5a949e932d4b1f7d6/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.425629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e1d34f03ca4c8a85eedb63a57311f2, user_id=7da7a5b617164ff5a949e932d4b1f7d6, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.428460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.430708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:09.491268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1203/2397] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:09 2022] PUT /identity/v3/projects/f4e1d34f03ca4c8a85eedb63a57311f2/users/7da7a5b617164ff5a949e932d4b1f7d6/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:09.503336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.503623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.503892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.533111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] MFA Rules not processed for user `7da7a5b617164ff5a949e932d4b1f7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:09.534274 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:09.600644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1195/2398] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:09.605250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.673103 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9e40-92f0-4155-bd69-8e2c6de95a04 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.677843 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.681673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.681673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.681673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.681673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.681673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.771323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1204/2399] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:09 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:09.775768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.787469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec66e8-38ff-4e4d-b906-6083f8b336af None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.844846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.845448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.845728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.846029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.846654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.848717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.946379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1196/2400] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:09 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:09.950406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.961558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44080fbf-6598-44ba-9c1d-e8e70f35c8b0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:09.966560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:09.967155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:09.968014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:09.968432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:09.969064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:09.970643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:09.981730 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1205/2401] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:41:09.986976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:09.997231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd6792-650d-4d71-bffc-67a974180691 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.001929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b53c0-39d8-45e8-b9a7-dce94ed5cad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.002516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b53c0-39d8-45e8-b9a7-dce94ed5cad4 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.002794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b53c0-39d8-45e8-b9a7-dce94ed5cad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.003064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b53c0-39d8-45e8-b9a7-dce94ed5cad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.003697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b53c0-39d8-45e8-b9a7-dce94ed5cad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.005520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b53c0-39d8-45e8-b9a7-dce94ed5cad4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.015995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1197/2402] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:10.020384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.031767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dae8bf-b411-425b-9ce0-f6f07ac92fa5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.036704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.037387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.037742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.037942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] PATH_INFO: `/v3/projects/026e2e5b55d446f68d52d9aeab8130b8/users/edb5122371024fbf975b4e7f0d45e069/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.054327 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=026e2e5b55d446f68d52d9aeab8130b8, user_id=edb5122371024fbf975b4e7f0d45e069, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.057056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.105827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1206/2403] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:10 2022] PUT /identity/v3/projects/026e2e5b55d446f68d52d9aeab8130b8/users/edb5122371024fbf975b4e7f0d45e069/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:41:10.117337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d941fce-90c9-4210-8f5a-5c4244a9778d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.117789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d941fce-90c9-4210-8f5a-5c4244a9778d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.118261 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d941fce-90c9-4210-8f5a-5c4244a9778d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.150728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3d941fce-90c9-4210-8f5a-5c4244a9778d None None] MFA Rules not processed for user `edb5122371024fbf975b4e7f0d45e069`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:10.151942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3d941fce-90c9-4210-8f5a-5c4244a9778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:10.181623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:10.219584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1198/2404] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:10.224093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.293936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ea8e-155f-402b-b6d8-682de378bf1d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.300444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.304285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.304285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.304285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.304285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.304707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.360484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81c73-19ee-4abc-ae66-024549ed8fd6 None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.360869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81c73-19ee-4abc-ae66-024549ed8fd6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.361217 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81c73-19ee-4abc-ae66-024549ed8fd6 None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.362655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1199/2405] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:41:10.368431 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.368606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.368878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.392007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:10.392738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:10.429478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1207/2406] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:10 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:10.433797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.475297 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:10.476599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1200/2407] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:41:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:41:10.488137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.489634 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:10.509664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a7f14-9183-45fd-b6b2-dace0c0a7cfd admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.514462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.515595 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.515987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.516260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.516936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.518857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.548308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab13687-c683-47f7-8fef-3cf5b32e059d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.596783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.597390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.597668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.598072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.606545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1208/2408] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:10 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:41:10.608411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.618943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323ba02-4cea-43ab-ab1d-62ce383d5f61 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.623585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.624911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.625152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.625454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.626197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.628228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.630746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:10.639693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1209/2409] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:41:10.644879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.656487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ca5f4-bff3-46bf-b722-ea3ccf379d70 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.660798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.661476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.661756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.662025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] PATH_INFO: `/v3/projects/a4f7c6264c9349fb8bb421b875037cae/users/4a356db342c54384b55d103290b5d087/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.673063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.674681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.677739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f7c6264c9349fb8bb421b875037cae, user_id=4a356db342c54384b55d103290b5d087, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.680451 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.685928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1201/2410] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:41:10.775495 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1210/2411] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:10 2022] PUT /identity/v3/projects/a4f7c6264c9349fb8bb421b875037cae/users/4a356db342c54384b55d103290b5d087/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:41:10.779624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.790854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe764a6-60f9-4fe2-93e7-cf76a1a93e76 service glance] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.851252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed57dd-a145-444c-acf8-34eab07dacb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.851814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed57dd-a145-444c-acf8-34eab07dacb3 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.852220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed57dd-a145-444c-acf8-34eab07dacb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.852670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed57dd-a145-444c-acf8-34eab07dacb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.853348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed57dd-a145-444c-acf8-34eab07dacb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.855018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed57dd-a145-444c-acf8-34eab07dacb3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.866414 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1202/2412] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:10.870636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.881244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71195201-b577-49a3-b85c-b936766cf08d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:10.884724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f8fe6-80e6-41d3-85af-fe24ba2fdcae None None] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.888077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f8fe6-80e6-41d3-85af-fe24ba2fdcae None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.888077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f8fe6-80e6-41d3-85af-fe24ba2fdcae None None] PATH_INFO: `/` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.888282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efe6da-ccd1-4084-adde-3e892a280a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:10.888489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1203/2413] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:41:10.888593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efe6da-ccd1-4084-adde-3e892a280a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.888593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efe6da-ccd1-4084-adde-3e892a280a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.888593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efe6da-ccd1-4084-adde-3e892a280a6f admin admin] PATH_INFO: `/v3/projects/a4f7c6264c9349fb8bb421b875037cae/users/4a356db342c54384b55d103290b5d087/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.889848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efe6da-ccd1-4084-adde-3e892a280a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f7c6264c9349fb8bb421b875037cae, user_id=4a356db342c54384b55d103290b5d087, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:10.891993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.892352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.892675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efe6da-ccd1-4084-adde-3e892a280a6f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:10.892801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.906953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1211/2414] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:10 2022] PUT /identity/v3/projects/a4f7c6264c9349fb8bb421b875037cae/users/4a356db342c54384b55d103290b5d087/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:10.917046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] MFA Rules not processed for user `b5f5987c024d44acbc7bf1502f11848f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:10.918034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07937-d7f9-46da-bb8c-f11bc4b97af6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:10.918270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:10.918382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07937-d7f9-46da-bb8c-f11bc4b97af6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:10.918579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07937-d7f9-46da-bb8c-f11bc4b97af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:10.949857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-aae07937-d7f9-46da-bb8c-f11bc4b97af6 None None] MFA Rules not processed for user `4a356db342c54384b55d103290b5d087`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:10.951115 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-aae07937-d7f9-46da-bb8c-f11bc4b97af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:10.971441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1204/2415] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:41:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:10.984713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:10.986115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:10.996406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b83e-d87f-4915-a83a-9c3df1f9caee None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:11.000380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:11.000795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:11.001070 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:11.001342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:11.008701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:11.010135 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:11.018956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1205/2416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:41:11.027412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1212/2417] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:41:11.039458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:11.040902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:11.099101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80969f69-6bc6-43e8-b8b7-01d70c8badbb service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:11.102875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:11.103416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:11.103698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:11.103972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:11.107990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254c01-a4bc-4488-a342-59b3ec195ba9 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:11.108263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254c01-a4bc-4488-a342-59b3ec195ba9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:11.108576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254c01-a4bc-4488-a342-59b3ec195ba9 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:11.109920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1213/2418] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:41:11.111788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:11.115746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:11.116129 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:11.116502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:11.137479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] MFA Rules not processed for user `f1427404350643b2892469b0da65b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:11.138696 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:11.170306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:11.171904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:11.181333 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1206/2419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:11.193036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:11.213068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1214/2420] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:41:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:11.225444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:11.235218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14f278-e4d6-43a5-8456-f8233ee15711 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:11.238962 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba2274-774c-4073-8d39-353203768c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:11.239562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba2274-774c-4073-8d39-353203768c57 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:11.239835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba2274-774c-4073-8d39-353203768c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:11.240100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba2274-774c-4073-8d39-353203768c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:11.250867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba2274-774c-4073-8d39-353203768c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:11.252302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba2274-774c-4073-8d39-353203768c57 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:11.263598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1207/2421] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:13.158498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:13.159955 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:13.172227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a2e3-c778-4359-974d-1970ed67f385 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:13.176107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:13.176747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:13.177023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:13.177292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:13.241314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:13.242911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:13.253635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1215/2422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:13.981051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:13.981586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:13.981586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:14.004545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:14.005764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:14.068725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1208/2423] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:41:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:14.081403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:14.082803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:14.154942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2165f1-15e4-442a-94c3-ce8b93f6a13a service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:14.160956 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:14.161602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:14.161832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:14.162106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:14.173167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:14.173365 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:14.181810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1216/2424] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:41:14.716805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:14.726930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492c917-c248-4d1c-a78d-b9dc74887cf8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:14.730598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33a60-1787-4e7a-b839-53a6f8403418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:14.731260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33a60-1787-4e7a-b839-53a6f8403418 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:14.731549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33a60-1787-4e7a-b839-53a6f8403418 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:14.731823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33a60-1787-4e7a-b839-53a6f8403418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:14.744372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33a60-1787-4e7a-b839-53a6f8403418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:14.745121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33a60-1787-4e7a-b839-53a6f8403418 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:14.759444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1209/2425] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:19.652853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:19.723448 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea43f3-8bee-4c61-a0db-c01bbcf798de service swift] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:19.728523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:19.729291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:19.729576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:19.729855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] PATH_INFO: `/v3/users/7da7a5b617164ff5a949e932d4b1f7d6` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:19.731009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da7a5b617164ff5a949e932d4b1f7d6)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:19.733018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:19.751869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:20.503610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da7a5b617164ff5a949e932d4b1f7d6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:20.517459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da7a5b617164ff5a949e932d4b1f7d6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:20.525522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da7a5b617164ff5a949e932d4b1f7d6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:20.619519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da7a5b617164ff5a949e932d4b1f7d6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:20.620238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da7a5b617164ff5a949e932d4b1f7d6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:20.627989 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1217/2426] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:19 2022] DELETE /identity/v3/users/7da7a5b617164ff5a949e932d4b1f7d6 => generated 0 bytes in 979 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:20.632744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:20.633293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:20.633293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:20.657816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:20.659042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:20.725578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1210/2427] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:20.736729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:20.738241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:20.802904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402f49-ad2c-48d9-bd8c-cf9d7dfef540 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:20.806758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:20.807460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:20.808555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:20.808829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:20.819747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:20.821275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:20.831424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1218/2428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:21.310599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:21.312170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:21.325619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d317-88b9-4b58-8498-cd2f59bedd7c None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:21.331016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:21.331814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:21.332203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:21.332604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] PATH_INFO: `/v3/projects/f4e1d34f03ca4c8a85eedb63a57311f2` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:21.333875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e1d34f03ca4c8a85eedb63a57311f2)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:21.336401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:21.368638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:21.491503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e1d34f03ca4c8a85eedb63a57311f2', 'request_id': 'req-21430935-7cac-4bd9-8479-aad74b05d574'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.497835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e1d34f03ca4c8a85eedb63a57311f2', 'request_id': 'req-21430935-7cac-4bd9-8479-aad74b05d574'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.505508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e1d34f03ca4c8a85eedb63a57311f2', 'request_id': 'req-21430935-7cac-4bd9-8479-aad74b05d574'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.579832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] The token cache is being invalidate because project f4e1d34f03ca4c8a85eedb63a57311f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:21.580123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.580695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.582191 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1211/2429] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:21 2022] DELETE /identity/v3/projects/f4e1d34f03ca4c8a85eedb63a57311f2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:21.586954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:21.598294 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcbd23-22b3-4b0c-85cb-96522fe9e42f service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:21.664712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:21.665383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:21.665662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:21.665935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] PATH_INFO: `/v3/users/edb5122371024fbf975b4e7f0d45e069` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:21.667142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb5122371024fbf975b4e7f0d45e069)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:21.669112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:21.852868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:21.858638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb5122371024fbf975b4e7f0d45e069'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.871926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb5122371024fbf975b4e7f0d45e069'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.879411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb5122371024fbf975b4e7f0d45e069'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.952454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb5122371024fbf975b4e7f0d45e069'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.953190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb5122371024fbf975b4e7f0d45e069'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:21.960691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1219/2430] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:21 2022] DELETE /identity/v3/users/edb5122371024fbf975b4e7f0d45e069 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:22.795028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:22.796830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:22.819896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430935-7cac-4bd9-8479-aad74b05d574 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:22.888676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:22.889319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:22.889604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:22.889939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] PATH_INFO: `/v3/projects/026e2e5b55d446f68d52d9aeab8130b8` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:22.893540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=026e2e5b55d446f68d52d9aeab8130b8)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:22.896152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:22.993881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026e2e5b55d446f68d52d9aeab8130b8', 'request_id': 'req-c6c39da0-6367-4874-8ec6-adbb4f348a18'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.000497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026e2e5b55d446f68d52d9aeab8130b8', 'request_id': 'req-c6c39da0-6367-4874-8ec6-adbb4f348a18'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.007406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026e2e5b55d446f68d52d9aeab8130b8', 'request_id': 'req-c6c39da0-6367-4874-8ec6-adbb4f348a18'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.105184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] The token cache is being invalidate because project 026e2e5b55d446f68d52d9aeab8130b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:23.105402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.106055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.107623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1212/2431] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:22 2022] DELETE /identity/v3/projects/026e2e5b55d446f68d52d9aeab8130b8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:23.111951 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:23.113492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:23.127473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b0f0c-cccb-41d5-840c-112ca644065a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:23.194286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:23.194937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:23.195219 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:23.195494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] PATH_INFO: `/v3/users/4a356db342c54384b55d103290b5d087` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:23.196691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a356db342c54384b55d103290b5d087)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:23.198648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:23.408034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a356db342c54384b55d103290b5d087'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.426423 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a356db342c54384b55d103290b5d087'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.434801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a356db342c54384b55d103290b5d087'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.518326 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a356db342c54384b55d103290b5d087'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.518937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1d6b7596-1674-4fdb-90d7-d2065bf871ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a356db342c54384b55d103290b5d087'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:23.530193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1220/2432] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:23 2022] DELETE /identity/v3/users/4a356db342c54384b55d103290b5d087 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:24.000527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.004817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:24.016693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c39da0-6367-4874-8ec6-adbb4f348a18 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.059755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:24.089900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.090547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.091356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.091630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] PATH_INFO: `/v3/projects/a4f7c6264c9349fb8bb421b875037cae` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.093066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f7c6264c9349fb8bb421b875037cae)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.095003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.214010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f7c6264c9349fb8bb421b875037cae', 'request_id': 'req-720a56bd-8d24-478b-a095-51038224089c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:24.220731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f7c6264c9349fb8bb421b875037cae', 'request_id': 'req-720a56bd-8d24-478b-a095-51038224089c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:24.227078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f7c6264c9349fb8bb421b875037cae', 'request_id': 'req-720a56bd-8d24-478b-a095-51038224089c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:24.290254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] The token cache is being invalidate because project a4f7c6264c9349fb8bb421b875037cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:24.290505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:24.291246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-720a56bd-8d24-478b-a095-51038224089c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:24.293094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1213/2433] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:23 2022] DELETE /identity/v3/projects/a4f7c6264c9349fb8bb421b875037cae => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:24.299222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.299847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.299847 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.323822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:24.325086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:24.359100 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:24.400633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1221/2434] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:24.402850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.403166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.403441 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.439197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:24.440460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:24.449242 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1214/2435] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:24.453406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.464891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0b664-8ff4-4214-a2b6-bac53bc3b1be None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.469577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.470743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.471021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.471430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.472080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:41:24.472411 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.474043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.491241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1222/2436] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:41:24.492378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.502306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9d637-9c63-43cd-a41e-5d28c4cfe3fb None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.507284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.507891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.508170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.508477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.509117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.511221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.593836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1215/2437] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:24 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:41:24.599061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.609587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069d41-76a1-43e8-a6f7-78b9e48f1e85 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.669160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.669775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.670058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.670533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.671234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.673427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.775846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1223/2438] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:24 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:24.780152 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.791545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471fd1e-a82d-4e62-a471-954a38778750 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.796294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.796928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.797208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.797546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.798504 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.800154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.817144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1216/2439] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:24.818314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.828886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6e69c-cd33-42d5-a3f3-dab29e9c7fa9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.834621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae34b13-8552-46dd-ac3a-7104a05e9335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.835233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae34b13-8552-46dd-ac3a-7104a05e9335 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.835508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae34b13-8552-46dd-ac3a-7104a05e9335 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.835775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae34b13-8552-46dd-ac3a-7104a05e9335 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.839568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae34b13-8552-46dd-ac3a-7104a05e9335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.841277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae34b13-8552-46dd-ac3a-7104a05e9335 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.855360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1224/2440] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:41:24.859813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:24.870676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525e04e-2bb0-4dff-ae29-03de07cd7803 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:24.875467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:24.876718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.877000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.877276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] PATH_INFO: `/v3/projects/3c6c298d8d5148628cc5afe1373700e5/users/13d877b902fb41969d9b5be0aa31bd96/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:24.892626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6c298d8d5148628cc5afe1373700e5, user_id=13d877b902fb41969d9b5be0aa31bd96, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:24.895528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:24.969345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1217/2441] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:24 2022] PUT /identity/v3/projects/3c6c298d8d5148628cc5afe1373700e5/users/13d877b902fb41969d9b5be0aa31bd96/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:24.980055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:24.980360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:24.980634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.010926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] MFA Rules not processed for user `13d877b902fb41969d9b5be0aa31bd96`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:25.012201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:25.080850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1225/2442] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:25.085317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.150812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80267-6b95-44f3-b3f3-45f0534b7f0f admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:25.155699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:25.156303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.156612 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.156853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.158304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:25.160195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:25.169674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:25.296830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1218/2443] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:25 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:25.301186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.312192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1abce-b4b4-417a-8aca-bed2e726d42d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:25.343117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:25.373023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:25.373773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.374027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.374301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.374937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:25.377009 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:25.479349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1226/2444] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:25 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:25.483367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.494790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b07e7f-5ea7-4bca-8c61-cf842f246a7d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:25.499767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:25.500399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.500682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.501500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.502319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:25.503920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:25.514417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1219/2445] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:25.525815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.535958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2578fb-ad40-4c36-b577-5ead4a0aa506 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:25.540766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c77a6-464a-41cc-8054-ad98f52e976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:25.541356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c77a6-464a-41cc-8054-ad98f52e976a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.541636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c77a6-464a-41cc-8054-ad98f52e976a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.541902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c77a6-464a-41cc-8054-ad98f52e976a admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.542532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c77a6-464a-41cc-8054-ad98f52e976a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:25.544112 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c77a6-464a-41cc-8054-ad98f52e976a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:25.554707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1227/2446] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:25.558874 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.569866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b922-6685-436a-abeb-0b6f23e038a0 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:25.574689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:25.575330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.575607 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.575880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] PATH_INFO: `/v3/projects/70555cc0682c495082d67aaf1537445f/users/1f6940c8255748b299a67b5ca8dc70af/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.591377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70555cc0682c495082d67aaf1537445f, user_id=1f6940c8255748b299a67b5ca8dc70af, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:25.594144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:25.655373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1220/2447] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:25 2022] PUT /identity/v3/projects/70555cc0682c495082d67aaf1537445f/users/1f6940c8255748b299a67b5ca8dc70af/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:25.722286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.722587 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.722859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.751898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] MFA Rules not processed for user `1f6940c8255748b299a67b5ca8dc70af`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:25.753213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:25.825669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1228/2448] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:25.830515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.893228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f447c2c-4af2-4ffa-9285-ca8902807df2 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:25.898018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:25.898635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:25.898946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:25.899196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:25.900087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:25.902046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:25.980601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1221/2449] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:25 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:25.981903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:25.993238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493f8ac-7da6-491f-a0c5-39d15cbb85f2 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:26.058691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:26.059304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.059582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.059854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.060537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:26.062478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:26.145511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1229/2450] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:25 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:26.149729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:26.161223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a1bf-fe9b-49e5-bbb1-9f1134094fee admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:26.165945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:26.166547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.166825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.167095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.167737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:26.169387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:26.181256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1222/2451] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:26.185733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:26.197050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dc936-1f21-4191-8db4-67d5fc64d66c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:26.201897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:26.202579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.202859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.203134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] PATH_INFO: `/v3/projects/c3b0ec742caf4b93ba886b5249485bba/users/85c26d8ffdd545f894655acf019e230a/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.218527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b0ec742caf4b93ba886b5249485bba, user_id=85c26d8ffdd545f894655acf019e230a, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:26.221344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:26.334744 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1230/2452] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:26 2022] PUT /identity/v3/projects/c3b0ec742caf4b93ba886b5249485bba/users/85c26d8ffdd545f894655acf019e230a/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:26.338926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:26.349594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f8f4a-c1ef-4f10-8128-c1921433d4d7 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:26.408036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a90d5b-7640-4e20-a57d-68a68146f45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:26.408424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a90d5b-7640-4e20-a57d-68a68146f45c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.408705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a90d5b-7640-4e20-a57d-68a68146f45c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.408980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a90d5b-7640-4e20-a57d-68a68146f45c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.409621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a90d5b-7640-4e20-a57d-68a68146f45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:26.411220 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a90d5b-7640-4e20-a57d-68a68146f45c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:26.421546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1223/2453] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:26.430622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:26.437363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:26.441249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47779d5f-c3c8-425f-9810-f29f5b29b8c5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:26.446052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:26.446694 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.446969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.447406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] PATH_INFO: `/v3/projects/c3b0ec742caf4b93ba886b5249485bba/users/85c26d8ffdd545f894655acf019e230a/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.449422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b0ec742caf4b93ba886b5249485bba, user_id=85c26d8ffdd545f894655acf019e230a, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:26.452205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:26.462659 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1231/2454] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:26 2022] PUT /identity/v3/projects/c3b0ec742caf4b93ba886b5249485bba/users/85c26d8ffdd545f894655acf019e230a/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:26.473661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.473980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.474264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.503357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] MFA Rules not processed for user `85c26d8ffdd545f894655acf019e230a`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:26.504624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:26.516421 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:26.573683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1224/2455] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:26.585862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:26.587931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:26.646720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7441c53-20a6-4480-9e18-020ef9924e94 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:26.650905 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:26.651515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:26.651792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:26.652062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:26.709337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:26.710760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:26.720440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1232/2456] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:28.768532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:28.770071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:28.783270 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd016e-a5a5-465d-a5ef-a3121406c05a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:28.787258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada307c-a53f-4796-b11b-8232413b8b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:28.787894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada307c-a53f-4796-b11b-8232413b8b76 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:28.788188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada307c-a53f-4796-b11b-8232413b8b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:28.788472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada307c-a53f-4796-b11b-8232413b8b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:28.850886 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada307c-a53f-4796-b11b-8232413b8b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:28.852364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada307c-a53f-4796-b11b-8232413b8b76 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:28.862942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1225/2457] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:30.287527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:30.296315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ceb870-1de8-49f3-8256-2df76b659f16 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:30.300764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:30.301368 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:30.301648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:30.301921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:30.311329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:30.312806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:30.322662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1233/2458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:33.246138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:33.246620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:33.246999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:33.267594 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:33.268836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:33.327882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1226/2459] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 00:41:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:33.340703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:33.346424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:33.431362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb22ea0-ca1c-43b4-a661-37e78c8d88bf service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:33.438479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea0011-e230-45c6-b730-b9b3dca8bf1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:33.438479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea0011-e230-45c6-b730-b9b3dca8bf1f service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:33.438479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea0011-e230-45c6-b730-b9b3dca8bf1f service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:33.438479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea0011-e230-45c6-b730-b9b3dca8bf1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:33.449183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea0011-e230-45c6-b730-b9b3dca8bf1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:33.450952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea0011-e230-45c6-b730-b9b3dca8bf1f service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:33.461533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1234/2460] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:41:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:41:34.968556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:35.023101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:35.034672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518bb75-6a60-4ad7-ad95-d687aa458e57 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:35.039195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:35.039817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:35.040091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:35.040386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] PATH_INFO: `/v3/users/13d877b902fb41969d9b5be0aa31bd96` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:35.041494 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d877b902fb41969d9b5be0aa31bd96)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:35.043439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:35.265166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d877b902fb41969d9b5be0aa31bd96'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:35.329252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d877b902fb41969d9b5be0aa31bd96'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:35.329885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d877b902fb41969d9b5be0aa31bd96'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:35.344947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d877b902fb41969d9b5be0aa31bd96'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:35.353495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d877b902fb41969d9b5be0aa31bd96'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:35.360260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1227/2461] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:34 2022] DELETE /identity/v3/users/13d877b902fb41969d9b5be0aa31bd96 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:41:35.365469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:35.366027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:35.366027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:35.386406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:35.387645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:35.405083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:35.451144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1235/2462] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:35.461659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:35.463120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:35.522173 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3449d6-4490-4e38-95d1-02a6b5f0fbcc admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:35.525951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:35.526552 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:35.526853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:35.527127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:35.536887 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:35.538322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:35.547923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1228/2463] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:35.970459 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:35.971921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:35.985445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fff5e-67ab-4256-8c4d-1fe8a901815c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:35.992382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:35.993066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:35.993432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:35.993704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] PATH_INFO: `/v3/projects/3c6c298d8d5148628cc5afe1373700e5` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:35.994829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6c298d8d5148628cc5afe1373700e5)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:35.996817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:36.102961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6c298d8d5148628cc5afe1373700e5', 'request_id': 'req-cd4524de-a89a-41e8-aee6-ff226c574806'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.180823 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6c298d8d5148628cc5afe1373700e5', 'request_id': 'req-cd4524de-a89a-41e8-aee6-ff226c574806'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.187085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6c298d8d5148628cc5afe1373700e5', 'request_id': 'req-cd4524de-a89a-41e8-aee6-ff226c574806'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.199437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] The token cache is being invalidate because project 3c6c298d8d5148628cc5afe1373700e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:36.199761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.200466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.202093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1236/2464] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:35 2022] DELETE /identity/v3/projects/3c6c298d8d5148628cc5afe1373700e5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:36.206108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:36.217221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e0198-5436-417d-a382-53315fe7340a service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:36.275224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:36.276009 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:36.276292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:36.276610 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] PATH_INFO: `/v3/users/1f6940c8255748b299a67b5ca8dc70af` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:36.277702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6940c8255748b299a67b5ca8dc70af)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:36.279720 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:36.295510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:36.462917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6940c8255748b299a67b5ca8dc70af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.569356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6940c8255748b299a67b5ca8dc70af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.570027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6940c8255748b299a67b5ca8dc70af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.582815 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6940c8255748b299a67b5ca8dc70af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.589631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6940c8255748b299a67b5ca8dc70af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:36.595586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1229/2465] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:36 2022] DELETE /identity/v3/users/1f6940c8255748b299a67b5ca8dc70af => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:37.010961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:37.012517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:37.026183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4524de-a89a-41e8-aee6-ff226c574806 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:37.087962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:37.088645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:37.089066 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:37.089237 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] PATH_INFO: `/v3/projects/70555cc0682c495082d67aaf1537445f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:37.090488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70555cc0682c495082d67aaf1537445f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:37.092427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:37.191290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:37.192856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70555cc0682c495082d67aaf1537445f', 'request_id': 'req-4b336255-b4dc-4931-849c-d7d972fd7557'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.293892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70555cc0682c495082d67aaf1537445f', 'request_id': 'req-4b336255-b4dc-4931-849c-d7d972fd7557'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.299817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70555cc0682c495082d67aaf1537445f', 'request_id': 'req-4b336255-b4dc-4931-849c-d7d972fd7557'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.311939 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] The token cache is being invalidate because project 70555cc0682c495082d67aaf1537445f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:37.312228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.313006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.314671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1237/2466] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:37 2022] DELETE /identity/v3/projects/70555cc0682c495082d67aaf1537445f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:37.318869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:37.320407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:37.334779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab19c8-45cc-438a-ba20-e7c5bd07f80c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:37.401640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:37.402273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:37.402545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:37.402813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] PATH_INFO: `/v3/users/85c26d8ffdd545f894655acf019e230a` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:37.403945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c26d8ffdd545f894655acf019e230a)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:37.405903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:37.574925 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c26d8ffdd545f894655acf019e230a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.654423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c26d8ffdd545f894655acf019e230a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.655044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c26d8ffdd545f894655acf019e230a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.668271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c26d8ffdd545f894655acf019e230a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.675339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-73489b77-dbed-408c-990d-8a4a9f5bd22e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c26d8ffdd545f894655acf019e230a'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:37.683071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1230/2467] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:41:37 2022] DELETE /identity/v3/users/85c26d8ffdd545f894655acf019e230a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:38.125948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.127427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:38.141440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b336255-b4dc-4931-849c-d7d972fd7557 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.201583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.202220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.202502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.202775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] PATH_INFO: `/v3/projects/c3b0ec742caf4b93ba886b5249485bba` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.203922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b0ec742caf4b93ba886b5249485bba)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.205845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:38.308202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b0ec742caf4b93ba886b5249485bba', 'request_id': 'req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:38.380545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b0ec742caf4b93ba886b5249485bba', 'request_id': 'req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:38.388510 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b0ec742caf4b93ba886b5249485bba', 'request_id': 'req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:38.400848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] The token cache is being invalidate because project c3b0ec742caf4b93ba886b5249485bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:41:38.401163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:38.401821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcde82e6-2b1e-4a54-9179-e7bb95fbd854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:41:38.403325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1238/2468] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:41:38 2022] DELETE /identity/v3/projects/c3b0ec742caf4b93ba886b5249485bba => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:38.408311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.408826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.408921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.429743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:38.430957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:38.440208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:38.495198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1231/2469] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:38.504209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.504522 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.516007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.526473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:38.527770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:38.535937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1239/2470] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:38.540300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.550551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e821c9-4dcc-4319-9bb1-f8eb414dc670 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.555849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.556530 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.556800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.557237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.557983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:41:38.558352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.560017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:38.573283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1232/2471] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:41:38.577745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.588229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17268-5064-4d04-97f7-4eb50ef8ee14 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.593167 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.594047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.594329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.597228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.597228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.597228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:38.691195 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1240/2472] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:41:38 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:38.696400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.707258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9fe99-3b44-4d72-9e8e-eb30f09331e1 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.764598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.765201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.765474 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.765742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.766361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.768280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:38.855693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1233/2473] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:41:38 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:41:38.860006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.871274 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceb800-15ea-4b82-abd7-2129063ff927 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.875994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.876628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.876907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.877176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.877815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.879401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:38.890010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1241/2474] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:38.895100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.905964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57e4f0-a9f3-46a1-8be9-52af5fa705ed admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.910721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f9953-8e2e-43de-adaa-35e4eb5269e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.911413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f9953-8e2e-43de-adaa-35e4eb5269e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.911674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f9953-8e2e-43de-adaa-35e4eb5269e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.911946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f9953-8e2e-43de-adaa-35e4eb5269e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.912650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f9953-8e2e-43de-adaa-35e4eb5269e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.915088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f9953-8e2e-43de-adaa-35e4eb5269e4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:38.926628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1234/2475] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:41:38.937815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:38.949309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460e7d-4e4d-49d7-bd54-df1dba6f908e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:38.954119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:38.954766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:38.955042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:38.955312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] PATH_INFO: `/v3/projects/4f17ea07f4eb4e1e9f875cdb6801b69c/users/4cdee3106dfb469c9b933ae6fa95569c/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:38.970138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f17ea07f4eb4e1e9f875cdb6801b69c, user_id=4cdee3106dfb469c9b933ae6fa95569c, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:38.972792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:39.066472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1242/2476] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:41:38 2022] PUT /identity/v3/projects/4f17ea07f4eb4e1e9f875cdb6801b69c/users/4cdee3106dfb469c9b933ae6fa95569c/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:41:39.072343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bacc9-ab5b-4db7-ba66-d5e339be70ea None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:39.072647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bacc9-ab5b-4db7-ba66-d5e339be70ea None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:39.072948 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bacc9-ab5b-4db7-ba66-d5e339be70ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:39.091899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a16bacc9-ab5b-4db7-ba66-d5e339be70ea None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:39.093136 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a16bacc9-ab5b-4db7-ba66-d5e339be70ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:39.155490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1235/2477] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:39.165870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:39.167633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:39.173193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:39.230163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10843546-f808-4e37-87d0-166045c297ba admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:39.233953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:39.234560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:39.234838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:39.235109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:39.245482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:39.246916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:39.257687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1243/2478] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:41:40.558195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af716970-9648-4d4a-80f0-213b09cf9eef None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:40.558736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af716970-9648-4d4a-80f0-213b09cf9eef None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:40.558736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af716970-9648-4d4a-80f0-213b09cf9eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:40.587440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-af716970-9648-4d4a-80f0-213b09cf9eef None None] MFA Rules not processed for user `4cdee3106dfb469c9b933ae6fa95569c`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:40.588689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-af716970-9648-4d4a-80f0-213b09cf9eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:40.597455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:40.658396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1236/2479] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:40.670087 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:40.671534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:40.729194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbea7cd-0e7b-4503-b673-7af55d672803 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:40.732782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:40.733377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:40.733652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:40.733923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:40.742765 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:40.744171 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:40.752936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1244/2480] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:42.440486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:42.441008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:42.441008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:42.461630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] MFA Rules not processed for user `4cdee3106dfb469c9b933ae6fa95569c`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:41:42.462831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:42.469792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1237/2481] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:41:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:41:45.704301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:45.706300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:45.786273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8b8fd-d3cc-458f-9e63-9dd862006281 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:45.792990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:45.793751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:45.794031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:45.794300 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:45.796216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:41:45.846666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:45.861320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:45.862923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17dc56-67e6-4c8c-b020-754db5123edf service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:45.872916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1245/2482] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:52.455231 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=26 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:56.060269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:03.594450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:03.604628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75e5a-705a-4e91-9f4b-f2cd8b095e94 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:03.608255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:03.608892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:03.609174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:03.609443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:03.611859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:03.680926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:03.682433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:03.693819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1238/2483] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:04.333497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c075d-cd96-4bd3-8593-1fbe676f7167 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:04.334270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c075d-cd96-4bd3-8593-1fbe676f7167 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:04.334270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c075d-cd96-4bd3-8593-1fbe676f7167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:04.355433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-bf4c075d-cd96-4bd3-8593-1fbe676f7167 None None] MFA Rules not processed for user `4cdee3106dfb469c9b933ae6fa95569c`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:04.356676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bf4c075d-cd96-4bd3-8593-1fbe676f7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:04.364174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1246/2484] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:08.649098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:08.650787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:08.663193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589316ad-6d3e-4ed3-9be2-16e5668b0a0b service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:08.667781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:08.668416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:08.668695 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:08.668968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:08.670804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:08.764124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:08.765838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:08.777868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1239/2485] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:42:12.489391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:12.489884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:12.490088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:12.509850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] MFA Rules not processed for user `022c4688d9ce4df9a99ae5fa84e1d871`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:12.513132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:12.518688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1247/2486] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:18.772852 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=56 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:21.099833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:21.108985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f809c1-ecef-46b6-a3cb-458992b72b1d service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:21.112896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:21.113537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:21.113782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:21.114048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:21.116003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:21.138108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:21.139955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:21.172299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1240/2487] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:21.264380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:21.275472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141c0c7-3dbe-464c-8b0a-432fc3e0e8fa None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:21.279577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:21.280184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:21.280492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:21.280763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:21.283078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:21.297169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:21.298853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9342672-73a8-4b1d-ac8a-3bc6149073c2 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:21.310468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1248/2488] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:23.808842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:23.821201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70230b40-a17d-41c9-a0ad-1d4848d2f814 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:23.826979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:23.827785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:23.828125 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:23.828792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:23.830789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:23.849051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:23.850910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3426a-d29f-48fc-9e8f-4998e8485cbd service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:23.877844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1241/2489] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:27.914978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7464a-f971-43c3-b53f-52cc98c76ed5 None None] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:27.917021 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7464a-f971-43c3-b53f-52cc98c76ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:27.917812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7464a-f971-43c3-b53f-52cc98c76ed5 None None] PATH_INFO: `/` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:27.920817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1249/2490] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 00:42:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:42:27.976665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:27.978483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:27.978957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:28.050444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] MFA Rules not processed for user `42f4dfa35f464a3f9cdbbc1e1ee4a980`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:28.054845 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:28.107079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7464a-f971-43c3-b53f-52cc98c76ed5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:28.110941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b4d7464a-f971-43c3-b53f-52cc98c76ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:28.132526 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=13 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:28.218293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1242/2491] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 00:42:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:28.219491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:28.221848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:28.241717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602884-410c-4bd0-9e55-703ccc277e57 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:28.269088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:28.269959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:28.271429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:28.271702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] PATH_INFO: `/v3/users/45422a67a78f4784abc0f18ad84778f3` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:28.274056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45422a67a78f4784abc0f18ad84778f3)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:28.278645 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:28.331677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7464a-f971-43c3-b53f-52cc98c76ed5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:28.346136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:28.347377 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:28.347795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:28.348464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:28.352234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:28.371963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:28.373738 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:28.386583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1250/2492] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:42:28.387649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:28.389073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:28.404796 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80e15-92cc-44de-b2d4-dfe42f869753 service placement] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:28.412593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:28.413440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:28.413860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:28.414277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:28.433138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:28.433138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:28.446205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1251/2493] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:28.531426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45422a67a78f4784abc0f18ad84778f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:28.609515 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45422a67a78f4784abc0f18ad84778f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:28.617430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45422a67a78f4784abc0f18ad84778f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:28.648207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45422a67a78f4784abc0f18ad84778f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:28.663251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1991b94f-5e58-407c-932b-753f9feeb68e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45422a67a78f4784abc0f18ad84778f3'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:28.674775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1243/2494] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:28 2022] DELETE /identity/v3/users/45422a67a78f4784abc0f18ad84778f3 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:29.439322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:29.457684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01cb40-2973-48f2-8a0b-41b7bb06bb51 service placement] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:29.575220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:29.576131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:29.576779 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:29.577250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] PATH_INFO: `/v3/projects/ec5893d9a55f4ed28f6579397d0d6f21` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:29.578635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5893d9a55f4ed28f6579397d0d6f21)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:29.582047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:29.754075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5893d9a55f4ed28f6579397d0d6f21', 'request_id': 'req-d717f21c-43ea-464a-9c87-1d41b835a41c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:29.839068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5893d9a55f4ed28f6579397d0d6f21', 'request_id': 'req-d717f21c-43ea-464a-9c87-1d41b835a41c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:29.849437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5893d9a55f4ed28f6579397d0d6f21', 'request_id': 'req-d717f21c-43ea-464a-9c87-1d41b835a41c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:29.878235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] The token cache is being invalidate because project ec5893d9a55f4ed28f6579397d0d6f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:29.878581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:29.880125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d717f21c-43ea-464a-9c87-1d41b835a41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:29.889540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1252/2495] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:29 2022] DELETE /identity/v3/projects/ec5893d9a55f4ed28f6579397d0d6f21 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:29.891736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:29.892213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:29.892294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:29.922885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:29.925269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:29.998724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:30.041569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1244/2496] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:30.052662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.053253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.057788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.096197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:30.098193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:30.117343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1253/2497] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:30.122404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:30.142213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e10ec-567c-40c7-97a0-e999e7e18a8b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:30.158107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:30.158976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.161349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.161967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.162864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:42:30.163359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:30.165712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:30.209575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1245/2498] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:42:30.211452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:30.219638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:30.230076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769781-60c6-4fab-8152-c03d25a9b3b5 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:30.248831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:30.249624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.250152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.250674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.251573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:30.254564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:30.427615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1254/2499] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:30 2022] POST /identity/v3/projects => generated 363 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:30.431777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:30.464377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea7f2-1e92-4bdc-a323-9a27cd6b4260 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:30.575093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:30.575719 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.576045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.576452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.577538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:30.579722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:30.719835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1246/2500] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:30 2022] POST /identity/v3/users => generated 282 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:30.724039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:30.755848 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651404b1-40ea-4913-ad05-bb12a721bd7d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:30.761310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:30.768844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.768844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.768844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.768844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:30.768844 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:30.789156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1255/2501] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:30.795307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:30.809725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31093c46-f248-4769-8e79-7cf7c30e58f5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:30.821597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cef1d7-5396-4461-965d-0e654012ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:30.822332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cef1d7-5396-4461-965d-0e654012ad1a admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.823492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cef1d7-5396-4461-965d-0e654012ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.823895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cef1d7-5396-4461-965d-0e654012ad1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.824929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cef1d7-5396-4461-965d-0e654012ad1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:30.832995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cef1d7-5396-4461-965d-0e654012ad1a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:30.864914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1247/2502] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:42:30.866201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:30.880472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9ba4-9351-4ae0-a2eb-5de951422474 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:30.895658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:30.896735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:30.897249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:30.897739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] PATH_INFO: `/v3/projects/54f30864d3304091b3f6d6ff4ee986b7/users/afb3a77a95214ea09c7015d62f7bc696/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:30.939725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f30864d3304091b3f6d6ff4ee986b7, user_id=afb3a77a95214ea09c7015d62f7bc696, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:30.942635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:31.052279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1256/2503] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:30 2022] PUT /identity/v3/projects/54f30864d3304091b3f6d6ff4ee986b7/users/afb3a77a95214ea09c7015d62f7bc696/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:31.066678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:31.067174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:31.067174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:31.117827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] MFA Rules not processed for user `afb3a77a95214ea09c7015d62f7bc696`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:31.120438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:31.247756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1248/2504] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:31.254256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:31.347795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578aec-0c00-4b5d-9d00-8339ea1571c0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:31.356914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:31.357810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:31.358257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:31.358714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:31.359683 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:31.362284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:31.377998 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:31.379301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:31.412309 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8c99a-96ef-413a-a1f5-0a18ce7baa97 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:31.432183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:31.432183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:31.432183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:31.432183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] PATH_INFO: `/v3/users/2a927401086b4a649919a6071b4a0fd9` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:31.432870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a927401086b4a649919a6071b4a0fd9)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:31.434213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:31.532496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1257/2505] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:31 2022] POST /identity/v3/projects => generated 363 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:31.539867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:31.625968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd4dc8-f04d-4f5c-a459-5aaec9e8b0ad admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:31.630776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:31.631486 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:31.631880 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:31.632248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:31.633181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:31.635795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:31.651099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:31.674028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a927401086b4a649919a6071b4a0fd9'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:31.747689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1258/2506] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:31 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:31.751869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:31.752358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a927401086b4a649919a6071b4a0fd9'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:31.753964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a927401086b4a649919a6071b4a0fd9'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:31.767249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5acd3a-18b2-4391-b370-80cc62fa08bd admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:31.771625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a927401086b4a649919a6071b4a0fd9'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:31.773777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:31.774867 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:31.775337 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:31.775758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:31.776665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:31.778407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:31.786488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a927401086b4a649919a6071b4a0fd9'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:31.791731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1259/2507] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:42:31.795685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1249/2508] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:31 2022] DELETE /identity/v3/users/2a927401086b4a649919a6071b4a0fd9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:42:31.796134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:31.798084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:31.812272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5326a-5145-4bf8-91ac-b15b25f31eb2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:31.818515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:31.882550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:31.883184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:31.883461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:31.883879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] PATH_INFO: `/v3/projects/26491d5a84d946df8607573fbf31144c/users/ceba53ca3f804f738d5dfac37cdb97dc/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:31.901915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26491d5a84d946df8607573fbf31144c, user_id=ceba53ca3f804f738d5dfac37cdb97dc, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:31.904822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:31.998381 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1260/2509] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:31 2022] PUT /identity/v3/projects/26491d5a84d946df8607573fbf31144c/users/ceba53ca3f804f738d5dfac37cdb97dc/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:32.002669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.018563 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ccf68-4f87-4115-a38f-05c2ac053bda admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:32.103016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:32.105226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.105675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.110495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.110495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:32.112277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.113606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:32.115533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:32.143729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466527c-65c6-40ec-b04a-e61896fa73c5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:32.154493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:32.161850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1250/2510] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:32.162280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.168020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.169973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.170260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] PATH_INFO: `/v3/projects/606c24bee18a4454ab672cfb6a395286` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.173024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=606c24bee18a4454ab672cfb6a395286)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:32.175613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:32.201378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3ae2e-86cf-40b4-b83f-96b57c1cfdbb admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:32.214604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f9a47-7f8a-4a8d-b6b4-dbb96557a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:32.215278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f9a47-7f8a-4a8d-b6b4-dbb96557a031 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.215747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f9a47-7f8a-4a8d-b6b4-dbb96557a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.216182 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f9a47-7f8a-4a8d-b6b4-dbb96557a031 admin admin] PATH_INFO: `/v3/projects/26491d5a84d946df8607573fbf31144c/users/ceba53ca3f804f738d5dfac37cdb97dc/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.218601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f9a47-7f8a-4a8d-b6b4-dbb96557a031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26491d5a84d946df8607573fbf31144c, user_id=ceba53ca3f804f738d5dfac37cdb97dc, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:32.221664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f9a47-7f8a-4a8d-b6b4-dbb96557a031 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:32.236416 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1251/2511] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:32 2022] PUT /identity/v3/projects/26491d5a84d946df8607573fbf31144c/users/ceba53ca3f804f738d5dfac37cdb97dc/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:32.253985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.254428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.254715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.335770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] MFA Rules not processed for user `ceba53ca3f804f738d5dfac37cdb97dc`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:32.342294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:32.348508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606c24bee18a4454ab672cfb6a395286', 'request_id': 'req-a120a074-ec15-4ca3-8918-ff3c13b509f5'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:32.422505 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606c24bee18a4454ab672cfb6a395286', 'request_id': 'req-a120a074-ec15-4ca3-8918-ff3c13b509f5'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:32.431466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606c24bee18a4454ab672cfb6a395286', 'request_id': 'req-a120a074-ec15-4ca3-8918-ff3c13b509f5'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:32.484616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] The token cache is being invalidate because project 606c24bee18a4454ab672cfb6a395286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:32.484616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:32.484616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a120a074-ec15-4ca3-8918-ff3c13b509f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:32.484616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1261/2512] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:32 2022] DELETE /identity/v3/projects/606c24bee18a4454ab672cfb6a395286 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:32.489120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57a69e-ff41-4c0d-84de-6c290ed8f573 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.489404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57a69e-ff41-4c0d-84de-6c290ed8f573 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.489678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57a69e-ff41-4c0d-84de-6c290ed8f573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.523617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1252/2513] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:32.533656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-3c57a69e-ff41-4c0d-84de-6c290ed8f573 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:32.533922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.541529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:32.542870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3c57a69e-ff41-4c0d-84de-6c290ed8f573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:32.592284 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:32.637203 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26748f86-ae18-478a-a5d0-371a336bd3c0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:32.643909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:32.643909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.644475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.645127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.658236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:32.659945 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:32.682366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1262/2514] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:32.684718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1253/2515] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:42:32.691784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.692083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.692546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.722157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:32.725039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:32.746756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1263/2516] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:42:32.763335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.778348 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536a15-0a31-4999-a331-19eefa6d2220 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:32.783979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:32.784790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.785317 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.785613 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.786311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:42:32.786619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:32.788386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:32.832636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1254/2517] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:42:32.833187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.847005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430b4d5-6c74-4bd0-9649-d50ee0ae3a1a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:32.857179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:32.857179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:32.857179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:32.857179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:32.857554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:32.858776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:32.865788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:32.973707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1264/2518] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:32 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:42:32.974928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:32.985812 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f1689-e541-4abc-9547-a300ea987585 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:33.044807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:33.045413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.045692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.045968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.046591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:33.048549 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:33.085831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:33.097948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:33.153890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1255/2519] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:32 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:33.160010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:33.188491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683baa25-7a0f-48d8-8da5-d58fd9b1c76d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:33.204712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:33.205365 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.205646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.205921 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.206690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:33.208389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:33.218414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de533ce-e17a-4be8-94c4-1d0ad63852db admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:33.227272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1256/2520] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:33.228693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:33.229363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.229647 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.229923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.231806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:33.232756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:33.247910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5fd86-da99-4659-9e85-3c483c12f751 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:33.257660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:33.258380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.258730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.259790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.260732 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:33.262491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:33.275283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1257/2521] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:42:33.281475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:33.309882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aba9f-9fed-4211-9b72-848482d73421 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:33.316308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:33.317087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.317374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.317649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] PATH_INFO: `/v3/projects/0e10d533d2554163adf398bba7f7acd1/users/7c0bb02b4eba4eb785114f6582b25a16/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.346221 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e10d533d2554163adf398bba7f7acd1, user_id=7c0bb02b4eba4eb785114f6582b25a16, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:33.349000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:33.363739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:33.365223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2690980-526f-4c7e-b6eb-a27983926c64 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:33.375068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1265/2522] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:33.471171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1258/2523] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:33 2022] PUT /identity/v3/projects/0e10d533d2554163adf398bba7f7acd1/users/7c0bb02b4eba4eb785114f6582b25a16/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:33.500785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.502904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.503255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.560473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] MFA Rules not processed for user `7c0bb02b4eba4eb785114f6582b25a16`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:33.561806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:33.721813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1266/2524] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:33.739923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:33.739923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:33.799166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:33.829546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df031605-809c-42d5-9832-1f1752cc9e48 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:33.835692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:33.836560 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:33.836851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:33.837806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:33.849519 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:33.849854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:33.860974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1259/2525] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:34.857020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:34.858813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:34.905703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:34.944218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af168ba2-a114-40f7-a835-9904b32d2f20 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:34.949756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:34.957750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:34.957750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:34.957750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] PATH_INFO: `/v3/users/c639977b5f6744b9939d1e5a9745cd03` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:34.957750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c639977b5f6744b9939d1e5a9745cd03)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:34.957750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:35.178700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c639977b5f6744b9939d1e5a9745cd03'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.192971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c639977b5f6744b9939d1e5a9745cd03'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.200557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c639977b5f6744b9939d1e5a9745cd03'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.260734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c639977b5f6744b9939d1e5a9745cd03'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.261543 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c639977b5f6744b9939d1e5a9745cd03'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.268697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1267/2526] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:34 2022] DELETE /identity/v3/users/c639977b5f6744b9939d1e5a9745cd03 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:35.563744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:35.584694 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb63b5-f36e-44c3-9d1e-2bdd2e9a6d67 service nova] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:35.654784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:35.655422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:35.655725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:35.655995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] PATH_INFO: `/v3/projects/28f0cda364854dfd9fde0249a751ecf4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:35.657170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f0cda364854dfd9fde0249a751ecf4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:35.659108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:35.764290 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f0cda364854dfd9fde0249a751ecf4', 'request_id': 'req-8a898546-4351-4ba6-a671-64094171473b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.769821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f0cda364854dfd9fde0249a751ecf4', 'request_id': 'req-8a898546-4351-4ba6-a671-64094171473b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.777095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f0cda364854dfd9fde0249a751ecf4', 'request_id': 'req-8a898546-4351-4ba6-a671-64094171473b'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.857273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] The token cache is being invalidate because project 28f0cda364854dfd9fde0249a751ecf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:35.857273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.857273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:35.857273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1260/2527] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:35 2022] DELETE /identity/v3/projects/28f0cda364854dfd9fde0249a751ecf4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:35.857721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:35.858443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:35.873616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf799ff-5537-41e2-b509-4cd6120655ca admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:35.933621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:35.934251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:35.934519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:35.934783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] PATH_INFO: `/v3/users/84bb2bd64d1348109d85415eda58f2c3` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:35.935904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84bb2bd64d1348109d85415eda58f2c3)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:35.937826 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:36.138266 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84bb2bd64d1348109d85415eda58f2c3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.144101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:36.153346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84bb2bd64d1348109d85415eda58f2c3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.160667 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84bb2bd64d1348109d85415eda58f2c3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.249183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84bb2bd64d1348109d85415eda58f2c3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.249898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84bb2bd64d1348109d85415eda58f2c3'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.257336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1268/2528] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:35 2022] DELETE /identity/v3/users/84bb2bd64d1348109d85415eda58f2c3 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:36.518648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:36.520196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:36.534083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a898546-4351-4ba6-a671-64094171473b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:36.596698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:36.597407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:36.597663 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:36.597936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] PATH_INFO: `/v3/projects/45c50656acc7484aa36ceb69fcd61f83` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:36.599088 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c50656acc7484aa36ceb69fcd61f83)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:36.601036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:36.610393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:36.722913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c50656acc7484aa36ceb69fcd61f83', 'request_id': 'req-42c82c15-94e0-411a-9597-61a49d3a4466'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.729294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c50656acc7484aa36ceb69fcd61f83', 'request_id': 'req-42c82c15-94e0-411a-9597-61a49d3a4466'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.736292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c50656acc7484aa36ceb69fcd61f83', 'request_id': 'req-42c82c15-94e0-411a-9597-61a49d3a4466'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.813314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] The token cache is being invalidate because project 45c50656acc7484aa36ceb69fcd61f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:36.813603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.814275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-42c82c15-94e0-411a-9597-61a49d3a4466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:36.815853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1261/2529] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:36 2022] DELETE /identity/v3/projects/45c50656acc7484aa36ceb69fcd61f83 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:38.453607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:38.459113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:38.469127 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74be628-ff93-45ff-8290-6d62028b8ba1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:38.538946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:38.539772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:38.540189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:38.540643 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] PATH_INFO: `/v3/users/afb3a77a95214ea09c7015d62f7bc696` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:38.542078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb3a77a95214ea09c7015d62f7bc696)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:38.544231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:38.743544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb3a77a95214ea09c7015d62f7bc696'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:38.755732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb3a77a95214ea09c7015d62f7bc696'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:38.763417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb3a77a95214ea09c7015d62f7bc696'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:38.838143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb3a77a95214ea09c7015d62f7bc696'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:38.839410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb3a77a95214ea09c7015d62f7bc696'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:38.847352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1269/2530] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:38 2022] DELETE /identity/v3/users/afb3a77a95214ea09c7015d62f7bc696 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:38.852047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7fe00-1e13-4a37-b1e1-e712774b2042 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:38.852591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7fe00-1e13-4a37-b1e1-e712774b2042 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:38.852726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7fe00-1e13-4a37-b1e1-e712774b2042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:38.873475 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e8d7fe00-1e13-4a37-b1e1-e712774b2042 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:38.874726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e8d7fe00-1e13-4a37-b1e1-e712774b2042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:38.936901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1262/2531] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:38.948045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:38.949906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:38.965425 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21534150-d451-49fc-8358-f78d580d6760 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:38.965712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21534150-d451-49fc-8358-f78d580d6760 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:38.965993 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21534150-d451-49fc-8358-f78d580d6760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:38.987368 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-21534150-d451-49fc-8358-f78d580d6760 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:38.988632 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-21534150-d451-49fc-8358-f78d580d6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:39.015742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:39.020613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f2236-1bfc-4c4a-828d-21bf31791199 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:39.025339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:39.026273 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:39.026753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:39.027034 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:39.039856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:39.039856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:39.051315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1270/2532] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:39.057438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1263/2533] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:42:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:42:39.249106 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:39.250616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:39.325285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dcee7-a917-4859-8ea8-32d709795782 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:39.330909 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:39.336714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:39.336714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:39.336714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:39.352559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:39.354726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:39.363787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1271/2534] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:42:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:42:39.648003 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21534150-d451-49fc-8358-f78d580d6760 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:39.651189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-21534150-d451-49fc-8358-f78d580d6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:39.688349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21534150-d451-49fc-8358-f78d580d6760 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:39.712406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:39.714121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:39.714544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:39.714819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] PATH_INFO: `/v3/projects/54f30864d3304091b3f6d6ff4ee986b7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:39.716113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f30864d3304091b3f6d6ff4ee986b7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:39.718831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:39.873969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:39.895629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f30864d3304091b3f6d6ff4ee986b7', 'request_id': 'req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:39.897757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f30864d3304091b3f6d6ff4ee986b7', 'request_id': 'req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:39.910352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f30864d3304091b3f6d6ff4ee986b7', 'request_id': 'req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:39.996849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] The token cache is being invalidate because project 54f30864d3304091b3f6d6ff4ee986b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:39.997439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.003866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.003866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1264/2535] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:39 2022] DELETE /identity/v3/projects/54f30864d3304091b3f6d6ff4ee986b7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:42:40.004825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:40.026493 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba9ae1-a70e-4170-a156-15d8f09d1210 service swift] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:40.137385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:40.138225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:40.138555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:40.138840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] PATH_INFO: `/v3/users/ceba53ca3f804f738d5dfac37cdb97dc` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:40.140196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceba53ca3f804f738d5dfac37cdb97dc)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:40.142975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:40.386136 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceba53ca3f804f738d5dfac37cdb97dc'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.399563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceba53ca3f804f738d5dfac37cdb97dc'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.407366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceba53ca3f804f738d5dfac37cdb97dc'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.517911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceba53ca3f804f738d5dfac37cdb97dc'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.518552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceba53ca3f804f738d5dfac37cdb97dc'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:40.521072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:40.526882 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1272/2536] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:40 2022] DELETE /identity/v3/users/ceba53ca3f804f738d5dfac37cdb97dc => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:40.650285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:40.651877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:40.711115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eadd32-d6c9-4023-8d5d-2bdf719ebb27 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:40.765899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:40.766724 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:40.766934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:40.767799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:40.769796 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:40.845155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:40.846611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28bd7-4348-487b-ae47-6fe057a1cfe9 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:40.857077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1265/2537] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:41.190504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:41.192376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:41.212367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b5d32-a889-4aae-b9fe-c76db9f2c799 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:41.218444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:41.218638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:41.218820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:41.219092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] PATH_INFO: `/v3/projects/26491d5a84d946df8607573fbf31144c` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:41.220247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26491d5a84d946df8607573fbf31144c)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:41.222247 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:41.330463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26491d5a84d946df8607573fbf31144c', 'request_id': 'req-e55ec27d-9da5-4a40-9794-c6dc355f3117'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:41.403150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26491d5a84d946df8607573fbf31144c', 'request_id': 'req-e55ec27d-9da5-4a40-9794-c6dc355f3117'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:41.409440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26491d5a84d946df8607573fbf31144c', 'request_id': 'req-e55ec27d-9da5-4a40-9794-c6dc355f3117'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:41.421803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] The token cache is being invalidate because project 26491d5a84d946df8607573fbf31144c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:41.422091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:41.422750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e55ec27d-9da5-4a40-9794-c6dc355f3117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:41.424241 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1273/2538] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:41 2022] DELETE /identity/v3/projects/26491d5a84d946df8607573fbf31144c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:41.429147 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:41.429629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:41.429946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:41.451272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:41.452559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:41.483279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:41.521658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1266/2539] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:41.531357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:41.531685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:41.531961 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:41.554060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:41.555307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:41.564197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1274/2540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:41.568617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:41.579476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c99fb-6d38-44b0-8e27-a3f60e29d8f0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:41.585535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:41.586210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:41.586491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:41.586759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:41.587417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:42:41.587713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:41.590080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:41.609085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1267/2541] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:42:41.610797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:41.621379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599d09b-fdfb-49dc-bc69-3841aadba8a7 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:41.626242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:41.626864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:41.627142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:41.627422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:41.628039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:41.629980 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:41.756140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1275/2542] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:41 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:41.756990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:41.767725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af9ad-971a-47d7-9d00-422b351f77e5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:41.832713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:41.833322 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:41.833603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:41.833875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:41.834506 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:41.836478 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:41.986974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1268/2543] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:41 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:42:41.991244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.006415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d8f18-92aa-47ac-a2c5-cfe72c2510ef admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.011977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.012621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.012901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.013227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.014159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.016117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.032430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1276/2544] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:42.035768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.048980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03acb5b-49b4-479f-9bed-07f85d890692 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.054866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b6985-05e6-4fa0-a7d1-17325460cf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.055529 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b6985-05e6-4fa0-a7d1-17325460cf68 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.055805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b6985-05e6-4fa0-a7d1-17325460cf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.056078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b6985-05e6-4fa0-a7d1-17325460cf68 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.057496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b6985-05e6-4fa0-a7d1-17325460cf68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.059260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b6985-05e6-4fa0-a7d1-17325460cf68 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.076366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1269/2545] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:42.078417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.092104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80ba0-1607-41e4-acbf-43eb77527547 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.097166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.097827 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.098109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.098382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] PATH_INFO: `/v3/projects/d324053387d148c3959f4a25e91469b7/users/269b9f5595274d87a0dcb69a66e5743e/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.117552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d324053387d148c3959f4a25e91469b7, user_id=269b9f5595274d87a0dcb69a66e5743e, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.120778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.226478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1277/2546] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:42 2022] PUT /identity/v3/projects/d324053387d148c3959f4a25e91469b7/users/269b9f5595274d87a0dcb69a66e5743e/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:42.239974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.240267 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.240611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.276255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] MFA Rules not processed for user `269b9f5595274d87a0dcb69a66e5743e`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:42.277961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:42.355483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1270/2547] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:42.360001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.450860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c123-39b7-4d29-9d06-492c57ff3cfa admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.456193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.457002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.457267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.457536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.458190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.460310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.604353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1278/2548] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:42 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:42:42.608384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.620148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79120c-a42b-444c-963d-e1ad34ad78e6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.644966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:42.679775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.680417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.680700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.680986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.681643 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.683698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.771163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1271/2549] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:42 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:42.772361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.780498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:42.786191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6dffa-535b-4238-80f0-8c5304ed7363 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.791729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.792352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.792641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.793028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.793665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.795260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.806773 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1279/2550] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:42:42.811343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.821511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64211c-3331-4d9b-848a-b5eaaff4b56b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.827849 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65402070-ed1c-4974-b61b-bb0d43b8ec65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.828830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65402070-ed1c-4974-b61b-bb0d43b8ec65 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.829156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65402070-ed1c-4974-b61b-bb0d43b8ec65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.829433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65402070-ed1c-4974-b61b-bb0d43b8ec65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.830325 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65402070-ed1c-4974-b61b-bb0d43b8ec65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.832104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65402070-ed1c-4974-b61b-bb0d43b8ec65 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:42.847647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1272/2551] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:42:42.847775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:42.859766 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b086e-57ad-45cd-ae62-d759b9ffd630 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:42.870690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:42.871388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:42.871808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:42.872085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] PATH_INFO: `/v3/projects/3f77f222bb004095a59d69101f1f1b90/users/0a128ecb179a4d97926b3e603dd27d35/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:42.912798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f77f222bb004095a59d69101f1f1b90, user_id=0a128ecb179a4d97926b3e603dd27d35, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:42.922393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:43.008465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1280/2552] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:42 2022] PUT /identity/v3/projects/3f77f222bb004095a59d69101f1f1b90/users/0a128ecb179a4d97926b3e603dd27d35/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:42:43.019952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:43.020343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:43.020727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:43.115413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] MFA Rules not processed for user `0a128ecb179a4d97926b3e603dd27d35`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:43.121113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:43.265004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1273/2553] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:43.269250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:43.436461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28eb5b0-858f-4a56-8cbd-622d9527c20e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:43.448519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:43.451232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:43.451577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:43.453371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:43.458869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:43.458869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:43.600043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1281/2554] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:43 2022] POST /identity/v3/projects => generated 387 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:43.601532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:43.620881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bff3a1-3297-4ace-9460-bd221e39b38a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:43.723937 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:43.724838 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:43.725215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:43.725627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:43.726352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:43.729427 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:43.864603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1274/2555] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:43 2022] POST /identity/v3/users => generated 300 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:43.866504 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:43.879176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347291b3-403a-4ec2-b111-bb1777b836e0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:43.885559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:43.886561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:43.887093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:43.887469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:43.888310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:43.890264 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:43.908102 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1282/2556] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:43.913146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:43.926394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de579a5-d9c6-4f51-ae10-f9b9d8e69279 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:43.932443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:43.934280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:43.934750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:43.935204 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] PATH_INFO: `/v3/projects/ed9f72a67b074118b011648e7b87d33b/users/2dfa0506a9a54e28995d3acc7dc1bc0d/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:43.952923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9f72a67b074118b011648e7b87d33b, user_id=2dfa0506a9a54e28995d3acc7dc1bc0d, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:43.957800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:44.029187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:44.037112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1275/2557] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:43 2022] PUT /identity/v3/projects/ed9f72a67b074118b011648e7b87d33b/users/2dfa0506a9a54e28995d3acc7dc1bc0d/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:44.048387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:44.058832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446205c-5c67-4298-ae43-85b647f5d55f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:44.177560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deafc8f-f6f3-4602-88d7-656de6976e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:44.178676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deafc8f-f6f3-4602-88d7-656de6976e7a admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:44.179148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deafc8f-f6f3-4602-88d7-656de6976e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:44.179582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deafc8f-f6f3-4602-88d7-656de6976e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:44.181038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deafc8f-f6f3-4602-88d7-656de6976e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:44.183229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deafc8f-f6f3-4602-88d7-656de6976e7a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:44.203388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1283/2558] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:44.207749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:44.222901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240b219-d103-42b8-9351-bc3cc79e7c46 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:44.229897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:44.230809 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:44.231092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:44.231364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] PATH_INFO: `/v3/projects/ed9f72a67b074118b011648e7b87d33b/users/2dfa0506a9a54e28995d3acc7dc1bc0d/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:44.233920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9f72a67b074118b011648e7b87d33b, user_id=2dfa0506a9a54e28995d3acc7dc1bc0d, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:44.236930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:44.266507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1276/2559] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:44 2022] PUT /identity/v3/projects/ed9f72a67b074118b011648e7b87d33b/users/2dfa0506a9a54e28995d3acc7dc1bc0d/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:44.278385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:44.278863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:44.278958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:44.372560 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] MFA Rules not processed for user `2dfa0506a9a54e28995d3acc7dc1bc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:44.374288 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:44.444596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:44.518817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1284/2560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:42:44.536936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:44.543794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:44.648968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6cdc9-7a4e-423a-bf2d-cb59de587f36 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:44.656561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:44.669565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:44.670219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:44.670700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:44.684740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:44.686741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:44.714934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1277/2561] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:46.318291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:46.321444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:46.427077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c88ea9-5e1b-4f38-aafc-36aeb5ddc92e None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:46.432740 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:46.440371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:46.440371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:46.440371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] PATH_INFO: `/v3/users/ac2476465c4c4bd2b1cf81934b6e9a7d` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:46.440371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2476465c4c4bd2b1cf81934b6e9a7d)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:46.440371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:46.652976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:46.655902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:46.670553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d96e9-b347-4a89-9c8e-3c2c6ae82632 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:46.683569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:46.684459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:46.684813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:46.685161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:46.756399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2476465c4c4bd2b1cf81934b6e9a7d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:46.775043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2476465c4c4bd2b1cf81934b6e9a7d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:46.785769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2476465c4c4bd2b1cf81934b6e9a7d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:46.792683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:46.794100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:46.805036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1278/2562] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:46.806115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:46.809175 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:46.861581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2476465c4c4bd2b1cf81934b6e9a7d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:46.862608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2476465c4c4bd2b1cf81934b6e9a7d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:46.874860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1285/2563] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:46 2022] DELETE /identity/v3/users/ac2476465c4c4bd2b1cf81934b6e9a7d => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:46.896956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e7cd3-cc2d-4c63-bdd7-9b6ac013a6da service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:46.997187 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:47.019234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:47.020008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:47.021283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:47.021283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:47.022605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:47.175890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:47.176807 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:47.208554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:47.210094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:47.221931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1279/2564] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:47.263295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:47.297496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cbcfd-a107-4511-b5e0-d161c1b8dc88 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:47.314382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:47.315128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:47.315508 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:47.316079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] PATH_INFO: `/v3/users/7c0bb02b4eba4eb785114f6582b25a16` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:47.317187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0bb02b4eba4eb785114f6582b25a16)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:47.319236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:47.364709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:47.371755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:47.393133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78111d49-f573-4efe-a746-b01027ac747c service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:47.402233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:47.402884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:47.404297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:47.404297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] PATH_INFO: `/v3/projects/5023455241594360bacff911a6641d7a` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:47.404674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5023455241594360bacff911a6641d7a)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:47.406743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:47.653236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0bb02b4eba4eb785114f6582b25a16'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.661271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5023455241594360bacff911a6641d7a', 'request_id': 'req-27e6acea-f012-4fde-9e76-e137e137be41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.675538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0bb02b4eba4eb785114f6582b25a16'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.676890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5023455241594360bacff911a6641d7a', 'request_id': 'req-27e6acea-f012-4fde-9e76-e137e137be41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.685507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0bb02b4eba4eb785114f6582b25a16'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.686462 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5023455241594360bacff911a6641d7a', 'request_id': 'req-27e6acea-f012-4fde-9e76-e137e137be41'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.799693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0bb02b4eba4eb785114f6582b25a16'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.801716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bc69f810-accf-4348-912c-5457662e1b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0bb02b4eba4eb785114f6582b25a16'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.838972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1286/2565] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:47 2022] DELETE /identity/v3/users/7c0bb02b4eba4eb785114f6582b25a16 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:47.849536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:47.850233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:47.850634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:47.885586 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] The token cache is being invalidate because project 5023455241594360bacff911a6641d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:47.885975 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.886733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-27e6acea-f012-4fde-9e76-e137e137be41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:47.889350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1280/2566] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:47 2022] DELETE /identity/v3/projects/5023455241594360bacff911a6641d7a => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:47.895554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de48a-e86d-4a1b-b32f-80dd40363a7c None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:47.895929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de48a-e86d-4a1b-b32f-80dd40363a7c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:47.896276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471de48a-e86d-4a1b-b32f-80dd40363a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:47.897922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:47.902960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:47.934238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-471de48a-e86d-4a1b-b32f-80dd40363a7c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:47.936334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-471de48a-e86d-4a1b-b32f-80dd40363a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:48.022609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1287/2567] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:48.037057 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.040566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:48.058595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1281/2568] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:42:48.067957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.068398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.068788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.120708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:48.143364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7f82e-b75b-4724-8cd4-d64237f61466 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.147820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.148828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.149721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.150539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.167453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.169539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.185725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1288/2569] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:48.233687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:48.235572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:48.334877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1282/2570] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:48.339697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.347439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:48.408383 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b498fc7-f039-4bec-9236-a1bfb3ec4377 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.415304 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.416056 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.416482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.416966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.417700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:42:48.418122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.420467 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.439712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1289/2571] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:42:48.444388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.459035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee08846-facb-45e1-81c0-2f035b89cda1 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.466083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.466717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.467126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.467260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.467930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.469894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.592248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1283/2572] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:48 2022] POST /identity/v3/projects => generated 357 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:42:48.594262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.607373 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e47772-da26-48b1-94e5-f2e060f1eec4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.716946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.717681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.717971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.718510 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.719152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.721525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.849998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1290/2573] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:48 2022] POST /identity/v3/users => generated 279 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:48.855502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.867359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe008685-1036-4eb2-92c9-965b37ddd0d9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.874037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.874759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.875040 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.875314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.875994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.877749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.892794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1284/2574] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:48.894191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.917764 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:48.918418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b6169-0efb-4fed-98dc-30b8a0f2c4cf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.920658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:48.938305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ddd7e-e7b1-4b52-932e-a6b0791ef56b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:48.943119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.943673 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.943963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.944238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.945029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.945700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:48.946533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:48.946684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.947229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:48.947964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] PATH_INFO: `/v3/projects/0e10d533d2554163adf398bba7f7acd1` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:48.951565 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e10d533d2554163adf398bba7f7acd1)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:48.953970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:48.971402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1291/2575] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:48.973618 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:49.003949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e11d67-1e85-4d44-8b0d-d2b7968afeba admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:49.017067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e0c7-f6d4-4b60-b54b-49d069b91ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:49.018047 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e0c7-f6d4-4b60-b54b-49d069b91ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.019058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e0c7-f6d4-4b60-b54b-49d069b91ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.019681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e0c7-f6d4-4b60-b54b-49d069b91ff5 admin admin] PATH_INFO: `/v3/projects/878cbed802814eb1930cb169ceb836cb/users/4b00798b6858425ea784d11e9307cc9d/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.049041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77e0c7-f6d4-4b60-b54b-49d069b91ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878cbed802814eb1930cb169ceb836cb, user_id=4b00798b6858425ea784d11e9307cc9d, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:49.055282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77e0c7-f6d4-4b60-b54b-49d069b91ff5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:49.192020 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1292/2576] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:48 2022] PUT /identity/v3/projects/878cbed802814eb1930cb169ceb836cb/users/4b00798b6858425ea784d11e9307cc9d/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:42:49.204494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.205398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.206419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.219559 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e10d533d2554163adf398bba7f7acd1', 'request_id': 'req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:49.227458 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e10d533d2554163adf398bba7f7acd1', 'request_id': 'req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:49.236375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e10d533d2554163adf398bba7f7acd1', 'request_id': 'req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:49.259792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] MFA Rules not processed for user `4b00798b6858425ea784d11e9307cc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:49.262146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:49.329055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] The token cache is being invalidate because project 0e10d533d2554163adf398bba7f7acd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:42:49.329392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:49.331172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-41c76ced-25fd-46c3-9f2b-5fdc8d7b42c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:49.338551 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1285/2577] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:42:48 2022] DELETE /identity/v3/projects/0e10d533d2554163adf398bba7f7acd1 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:49.342977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd416c-528b-4de6-b07d-b087fc4733fd None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.343879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd416c-528b-4de6-b07d-b087fc4733fd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.344363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd416c-528b-4de6-b07d-b087fc4733fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.390562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-74cd416c-528b-4de6-b07d-b087fc4733fd None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:49.391036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-74cd416c-528b-4de6-b07d-b087fc4733fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:49.398598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1293/2578] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:49.399825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:49.402181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:49.451818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:49.505712 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:49.570456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1286/2579] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:49.578998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f1b30-2534-4b59-8f86-e09af1cc8ce3 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:49.581276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.586102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.586102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.603445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:49.604726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.609093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.609093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.609093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:49.609093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:49.610223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:49.611922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:49.627620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1287/2580] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:42:49.632553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:49.648624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe1967-6f9c-4f65-a528-46435a2bbb79 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:49.655819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:49.656908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.657332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.657707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.659372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:42:49.659372 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:49.661884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:49.687139 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1288/2581] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:42:49.689176 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:49.706884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cf4ee-5900-42bd-9c31-4befc5dd583d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:49.718965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:49.725818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:49.727899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:49.730931 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:49.733617 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:49.736426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:49.758060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1294/2582] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/projects => generated 357 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:49.764657 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:49.873587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1289/2583] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:49.879041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:49.931233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649fba-4656-4ed7-8eb3-26f24b43ddfb admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.017988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cce402-6a3d-4a1a-80ac-c3e29ade2d19 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.043924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.044605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.044891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.045164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.046687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.049983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.081911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.083019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.083607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.084109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.085130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.089507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.491742 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1290/2584] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/users => generated 303 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:50.497123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.498478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1295/2585] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:49 2022] POST /identity/v3/users => generated 285 bytes in 735 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:50.499549 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.508736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f1c2-ce22-4391-9cb7-a6ca8c0fbf0e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.514271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.515111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.515254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.515409 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.516043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.521278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.521417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fef11-aeb4-49e0-a5b6-422ff03aa44a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.526842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.527863 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.528256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.528645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.529396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.530286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1291/2586] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:50.531364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.534422 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.544609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1296/2587] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:50.549189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.550214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc576c40-26d3-48f2-ba0e-16e6cb88e570 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.557517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.560128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.560128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.560128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.560128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.561172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.565218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b473a4-1709-45c0-b367-e23ad084591c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.578996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.580538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.581079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.581608 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] PATH_INFO: `/v3/projects/762d91965f324fcdb57456bd804553d0/users/78a56df4efe244c9b5011d740a77c1a4/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.595824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1292/2588] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:42:50.600524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.620030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a1633-7c25-4147-b38d-53ea5b66815c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.625027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:50.626311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6710-de95-4f40-912a-f1aeba9cdcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:50.627713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6710-de95-4f40-912a-f1aeba9cdcaa admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.628062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6710-de95-4f40-912a-f1aeba9cdcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.628187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762d91965f324fcdb57456bd804553d0, user_id=78a56df4efe244c9b5011d740a77c1a4, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.628505 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6710-de95-4f40-912a-f1aeba9cdcaa admin admin] PATH_INFO: `/v3/projects/a7b12ec00b9040818945df6705ad320c/users/7cfbd6a2d3904cf5a8e43e783dc57c9c/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.630929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.653829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d6710-de95-4f40-912a-f1aeba9cdcaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b12ec00b9040818945df6705ad320c, user_id=7cfbd6a2d3904cf5a8e43e783dc57c9c, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:50.662609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d6710-de95-4f40-912a-f1aeba9cdcaa admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:50.743401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1297/2589] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:50 2022] PUT /identity/v3/projects/762d91965f324fcdb57456bd804553d0/users/78a56df4efe244c9b5011d740a77c1a4/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:50.747540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.753120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1293/2590] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:50 2022] PUT /identity/v3/projects/a7b12ec00b9040818945df6705ad320c/users/7cfbd6a2d3904cf5a8e43e783dc57c9c/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:42:50.762424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:50.762983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:50.763404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:50.806213 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:50.808664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:50.874783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:50.937728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cee0d-4a37-497d-8a6b-814aaca616f4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:50.957759 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1294/2591] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:42:50.968711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:50.970443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:51.079272 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:51.080301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:51.080815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:51.081290 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:51.082197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:51.084357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:51.101979 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1298/2592] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:51.103943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:51.129601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527d88a-d41b-4302-bbca-c00e0509bb32 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:51.141392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726bd39-3055-476a-80c1-6eb1439d47fa admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:51.146351 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:51.146996 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:51.147266 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:51.147540 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:51.159601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7f3b9-20fc-49c0-91b7-c935ee06500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:51.160761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7f3b9-20fc-49c0-91b7-c935ee06500a admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:51.161407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7f3b9-20fc-49c0-91b7-c935ee06500a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:51.161926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7f3b9-20fc-49c0-91b7-c935ee06500a admin admin] PATH_INFO: `/v3/projects/762d91965f324fcdb57456bd804553d0/users/78a56df4efe244c9b5011d740a77c1a4/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:51.164450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7f3b9-20fc-49c0-91b7-c935ee06500a admin admin] RBAC: Authorizing `identity:create_grant(project_id=762d91965f324fcdb57456bd804553d0, user_id=78a56df4efe244c9b5011d740a77c1a4, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:51.168358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:51.169039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7f3b9-20fc-49c0-91b7-c935ee06500a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:51.169965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:51.192433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1299/2593] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:51 2022] PUT /identity/v3/projects/762d91965f324fcdb57456bd804553d0/users/78a56df4efe244c9b5011d740a77c1a4/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:51.192758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1295/2594] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:51.203490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:51.204295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:51.204592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:51.277645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] MFA Rules not processed for user `78a56df4efe244c9b5011d740a77c1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:51.278926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:51.376851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1300/2595] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:51.386324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:51.399968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:51.569516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ee5b6-48f6-4ab1-b7b5-b5fa1ea4cff1 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:51.673541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:51.674819 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:51.675107 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:51.675381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:51.692345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:51.693892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:51.741418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1296/2596] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:52.053416 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:52.065646 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cbbb7-8290-4edb-afb5-c00e6b70aa7d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:52.070716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3514-97f2-4b58-a19d-cb04831a9ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:52.079968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3514-97f2-4b58-a19d-cb04831a9ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:52.079968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3514-97f2-4b58-a19d-cb04831a9ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:52.079968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3514-97f2-4b58-a19d-cb04831a9ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:52.079968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:52.164809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3514-97f2-4b58-a19d-cb04831a9ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:52.167933 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3514-97f2-4b58-a19d-cb04831a9ae8 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:52.195782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1301/2597] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:52.962298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:52.976349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88ebc6-1b34-4ba7-8617-461c73d503cd service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:52.981972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:52.982901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:52.983347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:52.983824 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:52.986062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:53.002495 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:53.062729 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:53.063343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:53.063343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:53.067456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:53.069151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:53.099202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1297/2598] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:53.124886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] MFA Rules not processed for user `7cfbd6a2d3904cf5a8e43e783dc57c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:53.126374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:53.297123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1302/2599] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:53.302014 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:53.406568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197e88d-c0df-451b-a23b-eebe89363deb service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:53.411447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:53.412300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:53.412693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:53.412892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:53.413729 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:53.416477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:53.552159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1298/2600] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:42:53 2022] POST /identity/v3/projects => generated 405 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:53.557145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:53.574220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403d72a-09ab-4ef6-b996-7edadab1ce6d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:53.654718 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:53.655820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:53.656308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:53.656800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:53.657726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:53.660168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:53.768082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1303/2601] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:42:53 2022] POST /identity/v3/users => generated 309 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:42:53.769356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:53.789061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ed4e-db76-4cc6-90da-263ecce88f63 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:53.815493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:53.816301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:53.816572 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:53.822619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:53.822619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:53.822619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:53.836225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1299/2602] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:53.837470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:53.849764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb173d5-0d11-4c84-a2b6-4cb36007d316 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:53.855392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:53.856093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:53.856420 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:53.856698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] PATH_INFO: `/v3/projects/dbab1b2a4f7c4104b6ae99b62f2253a1/users/2ca6e2548bbf4098b0b4670aefcfbaf1/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:53.876835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbab1b2a4f7c4104b6ae99b62f2253a1, user_id=2ca6e2548bbf4098b0b4670aefcfbaf1, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:53.880232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:53.976929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:53.977506 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1304/2603] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:53 2022] PUT /identity/v3/projects/dbab1b2a4f7c4104b6ae99b62f2253a1/users/2ca6e2548bbf4098b0b4670aefcfbaf1/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:53.981689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:53.992781 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d79e8a-96b6-4fe8-bc7a-a5c4d94a45ad admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:54.081488 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9499f-de3f-4164-94e2-140d15c21c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:54.082121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9499f-de3f-4164-94e2-140d15c21c1e admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:54.082400 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9499f-de3f-4164-94e2-140d15c21c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:54.082672 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9499f-de3f-4164-94e2-140d15c21c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:54.083332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9499f-de3f-4164-94e2-140d15c21c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:54.085097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9499f-de3f-4164-94e2-140d15c21c1e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:54.105302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1300/2604] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:42:54.120497 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:54.136782 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2b79b-7898-44b9-a438-ec52ac683b50 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:54.143366 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:54.144312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:54.144845 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:54.145131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] PATH_INFO: `/v3/projects/dbab1b2a4f7c4104b6ae99b62f2253a1/users/2ca6e2548bbf4098b0b4670aefcfbaf1/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:54.177706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbab1b2a4f7c4104b6ae99b62f2253a1, user_id=2ca6e2548bbf4098b0b4670aefcfbaf1, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:54.177706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:54.177706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1305/2605] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:42:54 2022] PUT /identity/v3/projects/dbab1b2a4f7c4104b6ae99b62f2253a1/users/2ca6e2548bbf4098b0b4670aefcfbaf1/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:42:56.817509 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:56.818161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:56.818161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:56.848884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] MFA Rules not processed for user `2ca6e2548bbf4098b0b4670aefcfbaf1`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:42:56.850146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:56.863871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:56.976786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1301/2606] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:42:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:42:59.394614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:59.396632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:59.476150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:59.560062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657bbb1-91b1-4f45-83a1-d27975ec5ab3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:59.564585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090dee6-3ee5-48fa-b733-b02cbe763351 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:59.569379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:59.570202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:59.570497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:59.570770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] PATH_INFO: `/v3/users/7cfbd6a2d3904cf5a8e43e783dc57c9c` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:59.572657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cfbd6a2d3904cf5a8e43e783dc57c9c)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:59.575026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:59.580576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:59.581604 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:59.582080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:59.583590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] PATH_INFO: `/v3/users/269b9f5595274d87a0dcb69a66e5743e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:59.585548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269b9f5595274d87a0dcb69a66e5743e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:59.589752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:59.821280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cfbd6a2d3904cf5a8e43e783dc57c9c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.915681 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269b9f5595274d87a0dcb69a66e5743e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.923746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cfbd6a2d3904cf5a8e43e783dc57c9c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.924922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cfbd6a2d3904cf5a8e43e783dc57c9c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.939538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269b9f5595274d87a0dcb69a66e5743e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.953906 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269b9f5595274d87a0dcb69a66e5743e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.961685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cfbd6a2d3904cf5a8e43e783dc57c9c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.982978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-46fbd863-5a67-4873-bcf8-4d86d8eab470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cfbd6a2d3904cf5a8e43e783dc57c9c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:42:59.993207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1302/2607] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:59 2022] DELETE /identity/v3/users/7cfbd6a2d3904cf5a8e43e783dc57c9c => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:00.089734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269b9f5595274d87a0dcb69a66e5743e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.090283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269b9f5595274d87a0dcb69a66e5743e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.100424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1306/2608] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:42:59 2022] DELETE /identity/v3/users/269b9f5595274d87a0dcb69a66e5743e => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:00.108786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:00.109133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:00.109324 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:00.142867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:00.144946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:00.185797 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:00.248184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1303/2609] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:00.258320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:00.261052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:00.338339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:00.345650 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:00.371363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4aea23-af14-4bba-9ae1-508dfcae18e4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:00.380866 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:00.381621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:00.381967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:00.382249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:00.386788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82375c15-f604-4306-8ffd-a645c02868a0 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:00.395853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:00.396899 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:00.397373 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:00.397851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] PATH_INFO: `/v3/projects/a7b12ec00b9040818945df6705ad320c` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:00.399804 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7b12ec00b9040818945df6705ad320c)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:00.404083 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:00.417048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:00.419049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:00.433676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1307/2610] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:00.550915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7b12ec00b9040818945df6705ad320c', 'request_id': 'req-5727abd1-104b-4544-ad31-6be8fd037f7c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.559614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7b12ec00b9040818945df6705ad320c', 'request_id': 'req-5727abd1-104b-4544-ad31-6be8fd037f7c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.568079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7b12ec00b9040818945df6705ad320c', 'request_id': 'req-5727abd1-104b-4544-ad31-6be8fd037f7c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.643546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] The token cache is being invalidate because project a7b12ec00b9040818945df6705ad320c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:00.643794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.645190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:00.651059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1304/2611] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:00 2022] DELETE /identity/v3/projects/a7b12ec00b9040818945df6705ad320c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:00.655977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:00.667437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20f065-ac3d-4225-b3f7-88ff4849cf0d service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:00.676444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:00.764062 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:00.764800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:00.765077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:00.765349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] PATH_INFO: `/v3/users/2ca6e2548bbf4098b0b4670aefcfbaf1` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:00.766771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca6e2548bbf4098b0b4670aefcfbaf1)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:00.768816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:01.056269 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca6e2548bbf4098b0b4670aefcfbaf1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.090240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca6e2548bbf4098b0b4670aefcfbaf1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.107600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca6e2548bbf4098b0b4670aefcfbaf1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.147507 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:01.149404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:01.170817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727abd1-104b-4544-ad31-6be8fd037f7c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:01.180977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:01.180977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:01.180977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:01.180977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] PATH_INFO: `/v3/projects/d324053387d148c3959f4a25e91469b7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:01.181401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d324053387d148c3959f4a25e91469b7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:01.181401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:01.204099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca6e2548bbf4098b0b4670aefcfbaf1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.205648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca6e2548bbf4098b0b4670aefcfbaf1'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.219981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1308/2612] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:00 2022] DELETE /identity/v3/users/2ca6e2548bbf4098b0b4670aefcfbaf1 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:01.352312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd324053387d148c3959f4a25e91469b7', 'request_id': 'req-1ec07928-2c0b-423c-bbe2-53229135cbfa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.368593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd324053387d148c3959f4a25e91469b7', 'request_id': 'req-1ec07928-2c0b-423c-bbe2-53229135cbfa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.384133 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd324053387d148c3959f4a25e91469b7', 'request_id': 'req-1ec07928-2c0b-423c-bbe2-53229135cbfa'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.533977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] The token cache is being invalidate because project d324053387d148c3959f4a25e91469b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:01.534355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.536038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:01.540369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1305/2613] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:01 2022] DELETE /identity/v3/projects/d324053387d148c3959f4a25e91469b7 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:01.542624 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:01.544060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:01.624265 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d867-1c03-4f9c-a15d-22e316cf8b58 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:01.663176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:01.690915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:01.691492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:01.691775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:01.692073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] PATH_INFO: `/v3/users/0a128ecb179a4d97926b3e603dd27d35` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:01.693840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a128ecb179a4d97926b3e603dd27d35)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:01.695479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:01.709846 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:01.720485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:01.746517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec07928-2c0b-423c-bbe2-53229135cbfa admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:01.752970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:01.764547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:01.764547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:01.764547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] PATH_INFO: `/v3/projects/dbab1b2a4f7c4104b6ae99b62f2253a1` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:01.764547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbab1b2a4f7c4104b6ae99b62f2253a1)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:01.764547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:02.127584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:02.145567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a128ecb179a4d97926b3e603dd27d35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.146517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbab1b2a4f7c4104b6ae99b62f2253a1', 'request_id': 'req-1f0c0f6b-749d-41f4-96a4-913e07e42389'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.162216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a128ecb179a4d97926b3e603dd27d35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.165909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbab1b2a4f7c4104b6ae99b62f2253a1', 'request_id': 'req-1f0c0f6b-749d-41f4-96a4-913e07e42389'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.173734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a128ecb179a4d97926b3e603dd27d35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.174332 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbab1b2a4f7c4104b6ae99b62f2253a1', 'request_id': 'req-1f0c0f6b-749d-41f4-96a4-913e07e42389'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.237591 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a128ecb179a4d97926b3e603dd27d35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.238340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a128ecb179a4d97926b3e603dd27d35'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.246960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1309/2614] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:01 2022] DELETE /identity/v3/users/0a128ecb179a4d97926b3e603dd27d35 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:02.248407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:02.251278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:02.316268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] The token cache is being invalidate because project dbab1b2a4f7c4104b6ae99b62f2253a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:02.318676 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.321340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-1f0c0f6b-749d-41f4-96a4-913e07e42389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:02.330665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1306/2615] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:01 2022] DELETE /identity/v3/projects/dbab1b2a4f7c4104b6ae99b62f2253a1 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:02.339075 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f437-f66e-4070-8a9b-22fa30f40059 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:02.341518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f437-f66e-4070-8a9b-22fa30f40059 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:02.342184 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f437-f66e-4070-8a9b-22fa30f40059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:02.398919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-3687f437-f66e-4070-8a9b-22fa30f40059 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:02.407731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3687f437-f66e-4070-8a9b-22fa30f40059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:02.552498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1307/2616] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:02.563379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:02.564387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:02.565114 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:02.566858 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603df59-eeec-4e89-9a4b-f47638145632 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:02.576642 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:02.577846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:02.578410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:02.578894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:02.582207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:02.606046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:02.608920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:02.616278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:02.622231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:02.622901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1308/2617] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:02.624034 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:02.635106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8711f2b-88c7-4c38-aca0-2e1ccc5a90ca None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:02.642342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:02.643553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:02.644183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:02.644843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:02.646047 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:02.646558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:02.646747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1310/2618] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:02.648884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:02.671007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1309/2619] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:02.679382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:02.711069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde2551-7fb1-4797-8a18-a5c74ce14b71 service placement] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:02.718962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:02.720085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:02.720763 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:02.721378 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:02.722832 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:02.732012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:02.891224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1311/2620] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:02 2022] POST /identity/v3/projects => generated 363 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:02.897976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:02.916208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee193be3-c95b-43f2-8f0c-d7acaa08dea6 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.008583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.015086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.015561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.015913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.016707 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.019012 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.060225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.062414 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:03.073244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:03.084809 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47b4d-8c37-4c82-a000-cc5d615301fa admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.098027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.098697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.101538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.101538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] PATH_INFO: `/v3/projects/3f77f222bb004095a59d69101f1f1b90` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.101538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f77f222bb004095a59d69101f1f1b90)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.102557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.159490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1310/2621] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:02 2022] POST /identity/v3/users => generated 282 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:03.164041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.181983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f6bd6-65f9-413f-a053-d6028d945b15 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.187640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.188281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.188807 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.189084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.189917 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.191606 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.208687 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1311/2622] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:03.213358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.224350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325f4a8-0a3a-4000-9b35-32b3098864a0 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.232086 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.232813 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.233225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.233621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.234392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.236498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.248239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1312/2623] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:03.253161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.264168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b70f-082f-42f3-9605-bdd4f17df81c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.279479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5343a-709c-47ce-ad0e-a8a295003281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.279479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5343a-709c-47ce-ad0e-a8a295003281 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.279479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5343a-709c-47ce-ad0e-a8a295003281 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.279479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5343a-709c-47ce-ad0e-a8a295003281 admin admin] PATH_INFO: `/v3/projects/6b6fd3820b1a40cc9505e38754d32fe4/users/35f80005300241e48b2393b158c85a55/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.280011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:03.286971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f77f222bb004095a59d69101f1f1b90', 'request_id': 'req-63ea78de-51d4-49d5-8956-cb6feffac5a8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.287593 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5343a-709c-47ce-ad0e-a8a295003281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6fd3820b1a40cc9505e38754d32fe4, user_id=35f80005300241e48b2393b158c85a55, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.290361 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5343a-709c-47ce-ad0e-a8a295003281 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.419540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f77f222bb004095a59d69101f1f1b90', 'request_id': 'req-63ea78de-51d4-49d5-8956-cb6feffac5a8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.426716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f77f222bb004095a59d69101f1f1b90', 'request_id': 'req-63ea78de-51d4-49d5-8956-cb6feffac5a8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.439794 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] The token cache is being invalidate because project 3f77f222bb004095a59d69101f1f1b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:03.440001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.441068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.444849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1312/2624] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:03 2022] DELETE /identity/v3/projects/3f77f222bb004095a59d69101f1f1b90 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:03.448407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.449775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:03.454924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1313/2625] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:03 2022] PUT /identity/v3/projects/6b6fd3820b1a40cc9505e38754d32fe4/users/35f80005300241e48b2393b158c85a55/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:03.466283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.466683 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.467043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.499314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] MFA Rules not processed for user `35f80005300241e48b2393b158c85a55`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:03.500894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:03.530422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea78de-51d4-49d5-8956-cb6feffac5a8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.595113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1314/2626] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:03.600131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.601727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:03.606074 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.606808 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.607142 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.607544 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] PATH_INFO: `/v3/users/2dfa0506a9a54e28995d3acc7dc1bc0d` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.608803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dfa0506a9a54e28995d3acc7dc1bc0d)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.611748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.622546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103954d-b811-4e4e-823c-fe743fec8a8f None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.633004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.633943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.634276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.634698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.635923 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.639991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.754928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1315/2627] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:03 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:03.760762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.819703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:03.831897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dfa0506a9a54e28995d3acc7dc1bc0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.833118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821a431-428c-43f4-8729-639dcab62ff5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.838406 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.839013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.839297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.839646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.840356 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.842842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.847701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dfa0506a9a54e28995d3acc7dc1bc0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.857938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dfa0506a9a54e28995d3acc7dc1bc0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:03.964793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1316/2628] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:03 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:03.968631 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:03.979601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c472-61f0-4158-812a-66968e930e56 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:03.984689 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:03.985299 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:03.985578 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:03.985851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:03.986516 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:03.988130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:03.999257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dfa0506a9a54e28995d3acc7dc1bc0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.000784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dfa0506a9a54e28995d3acc7dc1bc0d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.003244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1317/2629] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:04.008740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.010038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1313/2630] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:03 2022] DELETE /identity/v3/users/2dfa0506a9a54e28995d3acc7dc1bc0d => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:04.012023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.037385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004c0c8-2c2c-4d16-95ea-b5fbdcd5339e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:04.069459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:04.100401 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:04.101081 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.104527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.104527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] PATH_INFO: `/v3/projects/8bbaf9777bb44dec89975bf98436ee92/users/121487409c83478fa56d5c4ef9b87992/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.123444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bbaf9777bb44dec89975bf98436ee92, user_id=121487409c83478fa56d5c4ef9b87992, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:04.126357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:04.229747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1318/2631] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:04 2022] PUT /identity/v3/projects/8bbaf9777bb44dec89975bf98436ee92/users/121487409c83478fa56d5c4ef9b87992/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:04.233757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.243946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8a6c9-f8ee-4d08-8324-20184c0c3cd8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:04.308369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412d7e8-9d94-4c17-b234-703731e92afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:04.308997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412d7e8-9d94-4c17-b234-703731e92afd admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.309619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412d7e8-9d94-4c17-b234-703731e92afd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.309777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412d7e8-9d94-4c17-b234-703731e92afd admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.310941 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412d7e8-9d94-4c17-b234-703731e92afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:04.312634 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412d7e8-9d94-4c17-b234-703731e92afd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:04.326181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1314/2632] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:04.329023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.340197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494abc1-79d8-4119-9556-10c36f264aa4 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:04.345970 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:04.346818 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.347749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.348023 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] PATH_INFO: `/v3/projects/8bbaf9777bb44dec89975bf98436ee92/users/121487409c83478fa56d5c4ef9b87992/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.350260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bbaf9777bb44dec89975bf98436ee92, user_id=121487409c83478fa56d5c4ef9b87992, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:04.354897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:04.366976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1319/2633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:04 2022] PUT /identity/v3/projects/8bbaf9777bb44dec89975bf98436ee92/users/121487409c83478fa56d5c4ef9b87992/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:04.383929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.383929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.383929 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.433131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] MFA Rules not processed for user `121487409c83478fa56d5c4ef9b87992`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:04.434405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.519059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1315/2634] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:04.531455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.533446 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.563387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.569234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.624514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:04.625810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e02ed-1ea8-4078-a6e0-47a11abfc7e9 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:04.630259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70e1bb-12fe-42b1-a838-6e805e5d5efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:04.631026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70e1bb-12fe-42b1-a838-6e805e5d5efc service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.631132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70e1bb-12fe-42b1-a838-6e805e5d5efc service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.631469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70e1bb-12fe-42b1-a838-6e805e5d5efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.643592 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70e1bb-12fe-42b1-a838-6e805e5d5efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:04.645019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70e1bb-12fe-42b1-a838-6e805e5d5efc service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:04.658103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1320/2635] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:04.663511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.663805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.664108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.672691 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e459f-72ab-4c03-9fc9-463c26ed5318 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:04.678496 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:04.679217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.679425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.679698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] PATH_INFO: `/v3/projects/ed9f72a67b074118b011648e7b87d33b` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.680879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9f72a67b074118b011648e7b87d33b)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:04.684238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:04.697834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:04.699103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.765616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1321/2636] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 00:43:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:04.778358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.779770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.820668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:04.823727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9f72a67b074118b011648e7b87d33b', 'request_id': 'req-5fb7f7e3-c14c-406c-8255-c9854f612b5d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.855599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e8b4f-f91d-4be2-bc7e-cfd83293d996 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:04.901367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9f72a67b074118b011648e7b87d33b', 'request_id': 'req-5fb7f7e3-c14c-406c-8255-c9854f612b5d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.913791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9f72a67b074118b011648e7b87d33b', 'request_id': 'req-5fb7f7e3-c14c-406c-8255-c9854f612b5d'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.920079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de425e0a-d084-40fd-8bb7-50bde33badbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:04.920741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de425e0a-d084-40fd-8bb7-50bde33badbe service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.921026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de425e0a-d084-40fd-8bb7-50bde33badbe service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.921302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de425e0a-d084-40fd-8bb7-50bde33badbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:04.930454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] The token cache is being invalidate because project ed9f72a67b074118b011648e7b87d33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:04.930874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.932393 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:04.934524 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1316/2637] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:04 2022] DELETE /identity/v3/projects/ed9f72a67b074118b011648e7b87d33b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:04.940031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:04.941430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:04.982961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de425e0a-d084-40fd-8bb7-50bde33badbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:04.984715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de425e0a-d084-40fd-8bb7-50bde33badbe service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:04.993619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1322/2638] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:43:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:43:04.995535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:04.995868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:04.996259 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.003029 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7f7e3-c14c-406c-8255-c9854f612b5d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.007126 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.007934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.008077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.008418 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.010917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:05.017788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:05.018914 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:05.073310 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.074743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbdeab-3c4c-4304-bc07-37cf05c5cb59 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.084454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1317/2639] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:05.085430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1323/2640] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:05.095117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.095404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.095687 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.117841 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:05.119932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:05.128441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1318/2641] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:05.132837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.143880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d226db-b44a-465a-b26e-1d3349244ec8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.148844 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.150417 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.150811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.150965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.151858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:05.152161 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.153803 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.167450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1324/2642] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:05.171638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.181205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c426a3-2724-4620-a9bb-c1bb2e54e97b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.186277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.191824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.191824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.191824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.191824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.191824 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.199974 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:05.274753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1319/2643] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:05 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:05.279755 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.286386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:05.292080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bab90-fb0f-459d-bc79-c8e21c8f0101 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.351262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.351860 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.352164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.352599 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.353424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.355520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.472468 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1325/2644] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:05 2022] POST /identity/v3/users => generated 301 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:05.476655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.488146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1f47-278e-4bb6-abcf-4d43ebc8ab9a admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.492947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.498082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.498082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.498082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.498082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.498082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.509108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1320/2645] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:05.513433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.523779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295a53-f3c6-4b09-a2ee-2fa8ea6a187b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.529680 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dcac9-61d4-407f-9b15-43324f6fc0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.530295 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dcac9-61d4-407f-9b15-43324f6fc0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.530573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dcac9-61d4-407f-9b15-43324f6fc0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.530841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dcac9-61d4-407f-9b15-43324f6fc0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.531512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dcac9-61d4-407f-9b15-43324f6fc0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.533201 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dcac9-61d4-407f-9b15-43324f6fc0d0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.547104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1326/2646] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:05.551395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.571550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc1a02-3ced-444f-9f7e-8102606566f4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.574731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.575415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.575704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.576424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] PATH_INFO: `/v3/projects/b8f9e54e2fdb475baaf7bfc58fa67be4/users/b3a406216fec4f60910935a6273cf6ae/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.594094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f9e54e2fdb475baaf7bfc58fa67be4, user_id=b3a406216fec4f60910935a6273cf6ae, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.605501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:05.763793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1321/2647] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:05 2022] PUT /identity/v3/projects/b8f9e54e2fdb475baaf7bfc58fa67be4/users/b3a406216fec4f60910935a6273cf6ae/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:05.775981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.776360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.776638 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.805711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] MFA Rules not processed for user `b3a406216fec4f60910935a6273cf6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:05.807777 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:05.876006 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1327/2648] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:05.880629 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:05.946397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9762958-7d7a-451e-98fe-9107be52b575 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:05.956064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:05.958343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:05.958343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:05.958343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:05.958343 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:05.959889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.035281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1322/2649] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:05 2022] POST /identity/v3/projects => generated 397 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:06.039830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:06.052832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef0c04-74c4-4e3f-94be-71caa652d57b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:06.114892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:06.115574 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.115854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.116126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.116939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:06.119545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.215738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1328/2650] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:06 2022] POST /identity/v3/users => generated 305 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:06.217640 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:06.232698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201616-7eb2-4f66-98ff-0bb52094895b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:06.240081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:06.240726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.241002 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.241271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.241915 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:06.244044 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.386757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1323/2651] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:06.391338 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:06.397079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:06.403426 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b12652-f208-43f8-bd8b-674b7b07d925 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:06.411484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:06.412156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.412455 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.416934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] PATH_INFO: `/v3/projects/66241f431548493e82e1d3143942f65f/users/ccd801530ae348f9b2bff021b0a8acc6/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.431028 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66241f431548493e82e1d3143942f65f, user_id=ccd801530ae348f9b2bff021b0a8acc6, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:06.433835 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.523894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1329/2652] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:06 2022] PUT /identity/v3/projects/66241f431548493e82e1d3143942f65f/users/ccd801530ae348f9b2bff021b0a8acc6/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:06.529222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:06.539202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71cc4e-498b-4d7a-8f7b-98bda306570e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:06.596990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ddcce-b94a-4b10-895c-02aabcfac73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:06.597582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ddcce-b94a-4b10-895c-02aabcfac73c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.597857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ddcce-b94a-4b10-895c-02aabcfac73c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.598124 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ddcce-b94a-4b10-895c-02aabcfac73c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.598887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ddcce-b94a-4b10-895c-02aabcfac73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:06.600514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ddcce-b94a-4b10-895c-02aabcfac73c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.613649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1324/2653] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:06.615527 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:06.632553 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a139ba-7907-447c-b3ec-3f9671079659 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:06.640646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:06.641790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.642225 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.642620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] PATH_INFO: `/v3/projects/66241f431548493e82e1d3143942f65f/users/ccd801530ae348f9b2bff021b0a8acc6/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.644974 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66241f431548493e82e1d3143942f65f, user_id=ccd801530ae348f9b2bff021b0a8acc6, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:06.653597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.665098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1330/2654] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:06 2022] PUT /identity/v3/projects/66241f431548493e82e1d3143942f65f/users/ccd801530ae348f9b2bff021b0a8acc6/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:06.676444 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.676851 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.677128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.706918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] MFA Rules not processed for user `ccd801530ae348f9b2bff021b0a8acc6`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:06.708116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:06.720582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:06.781754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1325/2655] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:06.793126 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:06.794853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:06.855794 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e228d2-9a9c-42f0-91f1-f86c25bd0f86 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:06.859644 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:06.860256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:06.860564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:06.860841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:06.870735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:06.872150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:06.882234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1331/2656] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:06 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:07.163210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:07.173389 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c447582-e18a-4279-9bef-678f500d7cc6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:07.177070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:07.177664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:07.177940 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:07.178206 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:07.236169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:07.237633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:07.247817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1326/2657] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:07 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:08.558646 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:08.560189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:08.630802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216fad6-3979-4374-b01b-1093a8b349fb service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:08.636481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:08.637377 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:08.637685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:08.638262 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] PATH_INFO: `/v3/users/4cdee3106dfb469c9b933ae6fa95569c` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:08.639352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cdee3106dfb469c9b933ae6fa95569c)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:08.641395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:08.948174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cdee3106dfb469c9b933ae6fa95569c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.038207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cdee3106dfb469c9b933ae6fa95569c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.038868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cdee3106dfb469c9b933ae6fa95569c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.053080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:09.054525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cdee3106dfb469c9b933ae6fa95569c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.063951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcf4613d-62a8-43f9-ae4b-33005f5d1907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cdee3106dfb469c9b933ae6fa95569c'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.070408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1332/2658] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:08 2022] DELETE /identity/v3/users/4cdee3106dfb469c9b933ae6fa95569c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:09.386026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:09.399711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eda1c9-448c-41df-9f6a-9b849ceb125d service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:09.463017 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:09.463767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:09.464361 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:09.464525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] PATH_INFO: `/v3/projects/4f17ea07f4eb4e1e9f875cdb6801b69c` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:09.465571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f17ea07f4eb4e1e9f875cdb6801b69c)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:09.467547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:09.627509 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f17ea07f4eb4e1e9f875cdb6801b69c', 'request_id': 'req-8d853959-1679-42de-af53-b6a4c48ed10c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.629138 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:09.723476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f17ea07f4eb4e1e9f875cdb6801b69c', 'request_id': 'req-8d853959-1679-42de-af53-b6a4c48ed10c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.730072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f17ea07f4eb4e1e9f875cdb6801b69c', 'request_id': 'req-8d853959-1679-42de-af53-b6a4c48ed10c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.741192 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] The token cache is being invalidate because project 4f17ea07f4eb4e1e9f875cdb6801b69c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:09.741480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.742110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-8d853959-1679-42de-af53-b6a4c48ed10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:09.744228 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1327/2659] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:09 2022] DELETE /identity/v3/projects/4f17ea07f4eb4e1e9f875cdb6801b69c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:09.753418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:09.754209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:09.754499 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:09.776876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:09.778162 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:09.845327 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1333/2660] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:09.853011 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:09.853791 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:09.854189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:09.874702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:09.875937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:09.884828 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1328/2661] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:09.889264 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:09.901800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6937e-02ee-46ba-a92b-d89dc7510fda None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:09.907995 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:09.908793 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:09.909068 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:09.909335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:09.909981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:09.910277 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:09.911882 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:09.932872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1334/2662] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:09.937312 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:09.949563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658d9bd-bf5a-413c-8922-60bcfe3b2f57 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:09.956716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:09.957759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:09.957759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:09.957983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:09.958737 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:09.960552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:10.079891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1329/2663] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:09 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:10.085855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:10.097604 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28afcf-4d95-4c55-b07b-17b8262a5d4d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:10.155856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:10.156413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:10.156705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:10.156982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:10.157622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:10.159658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:10.253671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1335/2664] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:10 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:10.257798 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:10.268912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fa207-b8c6-486a-8ff7-8cd6758e8cf4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:10.273748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:10.274622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:10.275645 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:10.275778 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:10.281850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:10.281850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:10.289871 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1330/2665] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:10.293988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:10.313115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c22bd7-0023-4e53-8f15-fcf01b256164 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:10.319342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d1b4d-3b99-4612-a896-523ec52c4a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:10.320043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d1b4d-3b99-4612-a896-523ec52c4a3d admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:10.320340 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d1b4d-3b99-4612-a896-523ec52c4a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:10.320620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d1b4d-3b99-4612-a896-523ec52c4a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:10.321293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d1b4d-3b99-4612-a896-523ec52c4a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:10.323930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d1b4d-3b99-4612-a896-523ec52c4a3d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:10.335194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1336/2666] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:10.341325 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:10.352291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8810e0b-d88a-4e7f-a761-1c7d11737adc admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:10.357677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:10.358476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:10.358757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:10.359028 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] PATH_INFO: `/v3/projects/644ca4166ff846fb944c2b39f0263abe/users/4c936369a3ff4138833cfc81a1104f20/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:10.375788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644ca4166ff846fb944c2b39f0263abe, user_id=4c936369a3ff4138833cfc81a1104f20, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:10.378597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:10.455315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1331/2667] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:10 2022] PUT /identity/v3/projects/644ca4166ff846fb944c2b39f0263abe/users/4c936369a3ff4138833cfc81a1104f20/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:10.460696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf81c-0865-43d6-a491-0bdeb6dab1e6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:10.461842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf81c-0865-43d6-a491-0bdeb6dab1e6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:10.461842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf81c-0865-43d6-a491-0bdeb6dab1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:10.483670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8c8bf81c-0865-43d6-a491-0bdeb6dab1e6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:10.485765 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8c8bf81c-0865-43d6-a491-0bdeb6dab1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:10.538981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:10.552163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1337/2668] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:10.566625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:10.566625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:10.654223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e496286-6b0a-4abb-a303-b6ad7548cc76 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:10.658050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:10.658588 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:10.658898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:10.659162 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:10.668908 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:10.670376 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:10.679912 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1332/2669] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:12.016448 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d6892-7a10-4202-8785-097aca2c7a6b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:12.016989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d6892-7a10-4202-8785-097aca2c7a6b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:12.016989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d6892-7a10-4202-8785-097aca2c7a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:12.050027 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4e5d6892-7a10-4202-8785-097aca2c7a6b None None] MFA Rules not processed for user `4c936369a3ff4138833cfc81a1104f20`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:12.051252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4e5d6892-7a10-4202-8785-097aca2c7a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:12.119424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1338/2670] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:12.130705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:12.132469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:12.194579 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56396d-8b86-4846-9dc2-9c98dc54f278 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:12.198854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:12.199454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:12.199728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:12.199995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:12.208314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:12.209754 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:12.215077 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:12.219576 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1333/2671] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:14.012128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afa5a9-989b-4cea-a805-35e83f784140 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:14.012696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afa5a9-989b-4cea-a805-35e83f784140 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:14.012696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afa5a9-989b-4cea-a805-35e83f784140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:14.034015 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-08afa5a9-989b-4cea-a805-35e83f784140 None None] MFA Rules not processed for user `4c936369a3ff4138833cfc81a1104f20`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:14.035263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-08afa5a9-989b-4cea-a805-35e83f784140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:14.043080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1339/2672] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:14.951893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:14.953803 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:14.969305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335330aa-0a3e-4bea-9168-4436f0b195a0 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:14.974121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:14.974767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:14.975037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:14.975301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] PATH_INFO: `/v3/users/b3a406216fec4f60910935a6273cf6ae` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:14.976475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a406216fec4f60910935a6273cf6ae)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:14.978562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:15.219988 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a406216fec4f60910935a6273cf6ae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:15.232923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a406216fec4f60910935a6273cf6ae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:15.240083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a406216fec4f60910935a6273cf6ae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:15.341070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a406216fec4f60910935a6273cf6ae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:15.342116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a406216fec4f60910935a6273cf6ae'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:15.351594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1334/2673] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:14 2022] DELETE /identity/v3/users/b3a406216fec4f60910935a6273cf6ae => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:15.355954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:15.355954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:15.355954 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:15.376754 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:15.379011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:15.448486 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1340/2674] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:15.469128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:15.470719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:15.537305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6e7a-3a03-483e-9d2f-46dc3e4523aa admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:15.541229 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:15.542527 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:15.542953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:15.543242 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:15.552967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:15.554465 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:15.565693 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1335/2675] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:16.035041 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:16.037076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:16.050392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874b1bd-0f71-42c6-b2e9-8be3f1458eec None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:16.055626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:16.056869 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:16.076816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:16.076816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] PATH_INFO: `/v3/projects/b8f9e54e2fdb475baaf7bfc58fa67be4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:16.076816 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f9e54e2fdb475baaf7bfc58fa67be4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:16.077828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:16.223245 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f9e54e2fdb475baaf7bfc58fa67be4', 'request_id': 'req-0f3e8a19-aa61-4285-bfa0-a473572b2980'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.234342 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f9e54e2fdb475baaf7bfc58fa67be4', 'request_id': 'req-0f3e8a19-aa61-4285-bfa0-a473572b2980'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.246470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f9e54e2fdb475baaf7bfc58fa67be4', 'request_id': 'req-0f3e8a19-aa61-4285-bfa0-a473572b2980'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.369335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:16.373685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] The token cache is being invalidate because project b8f9e54e2fdb475baaf7bfc58fa67be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:16.374010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.374653 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.376223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1341/2676] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:16 2022] DELETE /identity/v3/projects/b8f9e54e2fdb475baaf7bfc58fa67be4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:16.380580 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:16.392460 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfad23-2019-4113-a975-eda7a50733d1 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:16.491516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:16.491516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:16.491812 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:16.492116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] PATH_INFO: `/v3/users/ccd801530ae348f9b2bff021b0a8acc6` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:16.493557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd801530ae348f9b2bff021b0a8acc6)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:16.497781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:16.520972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:16.726535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd801530ae348f9b2bff021b0a8acc6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.739534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd801530ae348f9b2bff021b0a8acc6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.751186 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd801530ae348f9b2bff021b0a8acc6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.770534 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:16.772195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:16.786095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e8a19-aa61-4285-bfa0-a473572b2980 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:16.791030 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:16.792209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:16.792692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:16.793074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] PATH_INFO: `/v3/users/4b00798b6858425ea784d11e9307cc9d` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:16.794123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b00798b6858425ea784d11e9307cc9d)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:16.796062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:16.853422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd801530ae348f9b2bff021b0a8acc6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.854947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-bde25f93-b218-4322-807b-64040d48c767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd801530ae348f9b2bff021b0a8acc6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:16.862121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1336/2677] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:16 2022] DELETE /identity/v3/users/ccd801530ae348f9b2bff021b0a8acc6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:17.022911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b00798b6858425ea784d11e9307cc9d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.101334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b00798b6858425ea784d11e9307cc9d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.101967 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b00798b6858425ea784d11e9307cc9d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.118128 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b00798b6858425ea784d11e9307cc9d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.129545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b00798b6858425ea784d11e9307cc9d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.150011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1342/2678] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:16 2022] DELETE /identity/v3/users/4b00798b6858425ea784d11e9307cc9d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:17.156198 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.156473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.156764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.177120 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:17.178362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:17.242476 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1337/2679] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:17.256620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:17.257873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:17.321853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e9ed3-123e-43ba-92cf-7e060b6911fe admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:17.327198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e533-a5da-4f31-bab6-2efa4333fbcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:17.328007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e533-a5da-4f31-bab6-2efa4333fbcf service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.337141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e533-a5da-4f31-bab6-2efa4333fbcf service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.337141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e533-a5da-4f31-bab6-2efa4333fbcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.338038 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164e533-a5da-4f31-bab6-2efa4333fbcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:17.339621 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164e533-a5da-4f31-bab6-2efa4333fbcf service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:17.349668 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1343/2680] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:17.364982 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:17.366307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:17.380462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672db76f-35d2-4e82-894a-fd7cbd74c685 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:17.388950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:17.390685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.391025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.391222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] PATH_INFO: `/v3/projects/66241f431548493e82e1d3143942f65f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.392628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=66241f431548493e82e1d3143942f65f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:17.396088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:17.537270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66241f431548493e82e1d3143942f65f', 'request_id': 'req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.694156 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66241f431548493e82e1d3143942f65f', 'request_id': 'req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.702728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66241f431548493e82e1d3143942f65f', 'request_id': 'req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.746415 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] The token cache is being invalidate because project 66241f431548493e82e1d3143942f65f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:17.753342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.753342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a8b6b917-ef5d-4c04-8c50-45c1bb6700bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:17.753342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1338/2681] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:17 2022] DELETE /identity/v3/projects/66241f431548493e82e1d3143942f65f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:17.758489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.759255 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.759469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.783950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:17.785316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:17.852253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1344/2682] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:17.861196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.861480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.861753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.884367 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:17.885973 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:17.895076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1339/2683] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:17.897298 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:17.899183 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:17.903715 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:17.914283 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363ca5-6291-470f-a3c3-263f44052f78 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:17.915670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a1bbe-f138-4d61-8340-c2d28436682c None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:17.920250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:17.921082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.921455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.921820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.922777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:17.922989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:17.923616 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:17.924496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.925042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.928344 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] PATH_INFO: `/v3/projects/878cbed802814eb1930cb169ceb836cb` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.928466 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:17.928555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=878cbed802814eb1930cb169ceb836cb)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:17.929296 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:17.934901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:17.940471 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1340/2684] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:17.945492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:17.946222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:17.959286 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0c25a-9cae-4afb-a044-04742c8da7cf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:17.966322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:17.966322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:17.968721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:17.968721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:17.968721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:17.969611 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.054832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878cbed802814eb1930cb169ceb836cb', 'request_id': 'req-6848b3f6-cfde-47f9-8728-1d66a560f14d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.064172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878cbed802814eb1930cb169ceb836cb', 'request_id': 'req-6848b3f6-cfde-47f9-8728-1d66a560f14d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.069971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1341/2685] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:17 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:18.074965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.075670 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878cbed802814eb1930cb169ceb836cb', 'request_id': 'req-6848b3f6-cfde-47f9-8728-1d66a560f14d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.168202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] The token cache is being invalidate because project 878cbed802814eb1930cb169ceb836cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:18.168619 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.169496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.170226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e76c9-12b3-4ba7-92fa-62527f44e730 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:18.179810 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:18.180890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.181385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.182564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.183638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:18.184935 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1345/2686] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:17 2022] DELETE /identity/v3/projects/878cbed802814eb1930cb169ceb836cb => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:18.189224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.189507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.190950 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:18.264597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848b3f6-cfde-47f9-8728-1d66a560f14d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:18.269323 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:18.269963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.270293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.270738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] PATH_INFO: `/v3/users/78a56df4efe244c9b5011d740a77c1a4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.272007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a56df4efe244c9b5011d740a77c1a4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:18.274354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.290257 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1342/2687] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:18 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:18.295016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.298146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:18.312481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8559d7-5556-4e79-acdc-b03fdf1dad9f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:18.318039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:18.318925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.319208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.319678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.320756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:18.323487 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.341756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1343/2688] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:18.345727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.358771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03921f6-d6f3-48d9-b05d-caa804ce3d7b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:18.364119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:18.364759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.365037 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.365308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.365924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:18.367528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.387904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1344/2689] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:18.391055 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.402542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a550-2435-4f71-b659-060545a2e1c8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:18.408784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c8b56-0c35-49dd-8eec-3a010ee1a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:18.409462 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c8b56-0c35-49dd-8eec-3a010ee1a4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.409746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c8b56-0c35-49dd-8eec-3a010ee1a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.410081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c8b56-0c35-49dd-8eec-3a010ee1a4e2 admin admin] PATH_INFO: `/v3/projects/0184528a6c274737a4a70e87465e825b/users/b3e018fe263643b4960506aed7343006/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.428837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c8b56-0c35-49dd-8eec-3a010ee1a4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0184528a6c274737a4a70e87465e825b, user_id=b3e018fe263643b4960506aed7343006, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:18.431307 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c8b56-0c35-49dd-8eec-3a010ee1a4e2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.519787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a56df4efe244c9b5011d740a77c1a4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.540557 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1345/2690] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:18 2022] PUT /identity/v3/projects/0184528a6c274737a4a70e87465e825b/users/b3e018fe263643b4960506aed7343006/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:18.552078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.552363 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.552631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.585161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] MFA Rules not processed for user `b3e018fe263643b4960506aed7343006`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:18.586395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:18.592286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a56df4efe244c9b5011d740a77c1a4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.595106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a56df4efe244c9b5011d740a77c1a4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.631292 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a56df4efe244c9b5011d740a77c1a4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.650679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a56df4efe244c9b5011d740a77c1a4'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:18.655482 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1346/2691] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:18 2022] DELETE /identity/v3/users/78a56df4efe244c9b5011d740a77c1a4 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:18.685575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1346/2692] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:18.690223 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.692044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:18.716875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:18.784696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2aa225-d1a7-4c06-866d-1d09fdd7153a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:18.793265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:18.794025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:18.794501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:18.794783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:18.795423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:18.797727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:18.914481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1347/2693] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:18.918546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.929121 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:18.936891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:18.939577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:18.998022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5def4b-b153-4b69-8c5a-5fb871d1bb87 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.050035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ccc4-a2f6-4ec1-bd02-e447501de679 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.056799 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.058026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.058538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.059008 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] PATH_INFO: `/v3/users/35f80005300241e48b2393b158c85a55` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.060637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f80005300241e48b2393b158c85a55)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.062811 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.077837 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.078461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.078739 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.079081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.079758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.082535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.255276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1347/2694] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:18 2022] POST /identity/v3/users => generated 298 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:19.256577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.269997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782653-27d6-4b2e-aa5e-9d3b28e48843 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.282014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.283405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.283405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.283405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.283891 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.286759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.317277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1348/2695] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:19.319312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f80005300241e48b2393b158c85a55'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.323439 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.326401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.340977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d6347-44e1-4c02-a225-07c0f63fcbe3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.346321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.346874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.347213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.347555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] PATH_INFO: `/v3/projects/762d91965f324fcdb57456bd804553d0` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.348801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762d91965f324fcdb57456bd804553d0)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.351896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.408238 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f80005300241e48b2393b158c85a55'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.409389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f80005300241e48b2393b158c85a55'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.425648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f80005300241e48b2393b158c85a55'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.435528 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f80005300241e48b2393b158c85a55'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.443657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1348/2696] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:18 2022] DELETE /identity/v3/users/35f80005300241e48b2393b158c85a55 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:19.444756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.446460 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.499949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762d91965f324fcdb57456bd804553d0', 'request_id': 'req-daf5465e-137c-47db-ad9a-7e8fb97a0662'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.532396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e48f33-cad7-4f51-ac06-2c65efa16e9c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.592943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab730d-c4cf-44e8-b419-c7d26bf61539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.593802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab730d-c4cf-44e8-b419-c7d26bf61539 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.594099 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab730d-c4cf-44e8-b419-c7d26bf61539 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.594375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab730d-c4cf-44e8-b419-c7d26bf61539 admin admin] PATH_INFO: `/v3/projects/a2fb93b6f8554f9cbc9c22400d86f6c7/users/8d8ef841ac63474aa89e234231b91edf/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.614789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab730d-c4cf-44e8-b419-c7d26bf61539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2fb93b6f8554f9cbc9c22400d86f6c7, user_id=8d8ef841ac63474aa89e234231b91edf, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.615123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762d91965f324fcdb57456bd804553d0', 'request_id': 'req-daf5465e-137c-47db-ad9a-7e8fb97a0662'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.617496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab730d-c4cf-44e8-b419-c7d26bf61539 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.622784 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762d91965f324fcdb57456bd804553d0', 'request_id': 'req-daf5465e-137c-47db-ad9a-7e8fb97a0662'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.634473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] The token cache is being invalidate because project 762d91965f324fcdb57456bd804553d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:19.634759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.636348 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-daf5465e-137c-47db-ad9a-7e8fb97a0662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:19.638023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1349/2697] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:19 2022] DELETE /identity/v3/projects/762d91965f324fcdb57456bd804553d0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:19.639223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494b9f-721a-49ce-bb58-6ed7a6ad6fc3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.639499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494b9f-721a-49ce-bb58-6ed7a6ad6fc3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.639760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494b9f-721a-49ce-bb58-6ed7a6ad6fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.660189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f5494b9f-721a-49ce-bb58-6ed7a6ad6fc3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:19.661429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f5494b9f-721a-49ce-bb58-6ed7a6ad6fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.669351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1350/2698] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:19.670515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.670790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.671054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.699633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:19.700888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.712398 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1351/2699] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:19.713331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.731430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.734122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1349/2700] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:19 2022] PUT /identity/v3/projects/a2fb93b6f8554f9cbc9c22400d86f6c7/users/8d8ef841ac63474aa89e234231b91edf/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:19.735783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.737752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.737752 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.760795 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:19.762190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.804578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7ad6-8b03-475b-8476-31370875c4fc None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.847117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1350/2701] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:19.849375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.850758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:19.863389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e974c-2ca1-4f06-bd36-09d0f83b60c3 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.868216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.868867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.869142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.869419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.870068 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.871659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.880983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.881703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.882234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1351/2702] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:19.882721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.883004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.883712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.891231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:19.898823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a4e69-dfd9-4bad-ae16-7d302559a7cb admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.902374 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.903551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.904919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.906780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.906780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.906780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.907073 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:19.907166 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.908867 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.914855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1352/2703] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:19.915534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.926153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1352/2704] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:19.928140 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44691a5-9446-45ec-8750-3729b8790eb4 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.930727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:19.934946 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e6bab-8b06-4c91-ad9f-f8e11b73956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.935621 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6bab-8b06-4c91-ad9f-f8e11b73956d admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.935900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6bab-8b06-4c91-ad9f-f8e11b73956d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.936741 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6bab-8b06-4c91-ad9f-f8e11b73956d admin admin] PATH_INFO: `/v3/projects/a2fb93b6f8554f9cbc9c22400d86f6c7/users/8d8ef841ac63474aa89e234231b91edf/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.938463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e6bab-8b06-4c91-ad9f-f8e11b73956d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2fb93b6f8554f9cbc9c22400d86f6c7, user_id=8d8ef841ac63474aa89e234231b91edf, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.941554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e6bab-8b06-4c91-ad9f-f8e11b73956d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.941711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574fdf-0df5-46dc-9597-93e4c9fd3810 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:19.950965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:19.952134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:19.952853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.953337 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.953616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:19.954343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:19.955761 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1353/2705] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:19 2022] PUT /identity/v3/projects/a2fb93b6f8554f9cbc9c22400d86f6c7/users/8d8ef841ac63474aa89e234231b91edf/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:19.956276 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:19.969993 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:19.970421 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:19.970757 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.003669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] MFA Rules not processed for user `8d8ef841ac63474aa89e234231b91edf`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:20.005060 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:20.079155 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1353/2706] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:19 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:20.080172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.092949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1354/2707] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:20.104710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.106726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:20.158743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95517895-eda9-4c51-902c-181998ffca4d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.165369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.166167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.166490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.166984 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.167903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.169891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.212355 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8eab48-4f13-4e23-af1a-cfe75672a211 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.263429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.264084 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.264401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.264702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.280094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1354/2708] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:20 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:20.284829 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.297912 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91af206-5ca7-49bb-91c1-3d2ce19a2693 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.302848 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.303656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.303987 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.304378 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.304936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.306841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.321654 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1355/2709] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:20.325883 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.336997 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e13aae-2dc1-432d-9590-f66aa6c6b8c2 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.350532 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.351159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.351438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.351708 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.352533 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.354872 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.358150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.359617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.366892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1356/2710] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:20.382279 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1355/2711] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:20.386909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.426056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086cf2e-cdd9-4d95-85ab-1c3a0cbe3033 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.443541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d8fe3-b201-413d-87c7-39808753b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.443541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d8fe3-b201-413d-87c7-39808753b5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.443541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d8fe3-b201-413d-87c7-39808753b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.443541 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d8fe3-b201-413d-87c7-39808753b5d2 admin admin] PATH_INFO: `/v3/projects/8281363ad1c64e6393af42793f9d55c7/users/a3e774303f804a818739ad09b7bfc73c/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.481571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d8fe3-b201-413d-87c7-39808753b5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8281363ad1c64e6393af42793f9d55c7, user_id=a3e774303f804a818739ad09b7bfc73c, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.486955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d8fe3-b201-413d-87c7-39808753b5d2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.515707 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.525617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d3d77-e485-4dea-a81d-d164dc11ffa4 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.532623 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.533267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.533584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.533849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.535917 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:20.561744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1357/2712] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:20 2022] PUT /identity/v3/projects/8281363ad1c64e6393af42793f9d55c7/users/a3e774303f804a818739ad09b7bfc73c/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:20.577035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.577353 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.577623 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.620280 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] MFA Rules not processed for user `a3e774303f804a818739ad09b7bfc73c`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:20.621901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:20.660083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.661535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.671962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1356/2713] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:20.705805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1358/2714] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:20.711472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.738347 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347e92a-9703-4ec9-9815-c6532b5073e4 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.744635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:20.746639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:20.825682 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.826321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.826662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.826964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.827708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.829936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.865831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc00555-1d98-4baa-bc46-9b355b1241d8 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:20.873283 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:20.874020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:20.874402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:20.875120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] PATH_INFO: `/v3/projects/6b6fd3820b1a40cc9505e38754d32fe4` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:20.876039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6fd3820b1a40cc9505e38754d32fe4)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:20.878049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:20.954133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1357/2715] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:20 2022] POST /identity/v3/projects => generated 383 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:20.958502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.036785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250c382-4450-4e2a-92d1-ae382f91d612 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.038313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6fd3820b1a40cc9505e38754d32fe4', 'request_id': 'req-909a264a-bb7d-4259-a363-2154124d69e5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.045116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6fd3820b1a40cc9505e38754d32fe4', 'request_id': 'req-909a264a-bb7d-4259-a363-2154124d69e5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.051510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6fd3820b1a40cc9505e38754d32fe4', 'request_id': 'req-909a264a-bb7d-4259-a363-2154124d69e5'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.088042 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:21.096750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.097484 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.097626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.097897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.098531 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.100461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.126110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] The token cache is being invalidate because project 6b6fd3820b1a40cc9505e38754d32fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:21.126110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.126110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.126110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1359/2716] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:20 2022] DELETE /identity/v3/projects/6b6fd3820b1a40cc9505e38754d32fe4 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:21.128735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.130637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:21.143624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a264a-bb7d-4259-a363-2154124d69e5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.148556 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.149447 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.149727 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.150001 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] PATH_INFO: `/v3/users/121487409c83478fa56d5c4ef9b87992` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.151740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=121487409c83478fa56d5c4ef9b87992)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.153939 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.175479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1358/2717] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:20 2022] POST /identity/v3/users => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:21.179710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.181541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:21.193708 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a50ca-d893-4411-b12f-519ebb771290 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.196681 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:21.199335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.200149 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.200517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.200976 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.201552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.203148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.214649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1359/2718] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:21.220093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.231479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75cba8-5732-44e7-8f43-ac1e056f5d5c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.236428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.237073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.237394 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.237644 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] PATH_INFO: `/v3/projects/d1a3b92f544949d8a813d902fdf212d9/users/611cd33d05c94c60a11090e01b032d57/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.252874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a3b92f544949d8a813d902fdf212d9, user_id=611cd33d05c94c60a11090e01b032d57, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.255581 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.333301 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1360/2719] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:21 2022] PUT /identity/v3/projects/d1a3b92f544949d8a813d902fdf212d9/users/611cd33d05c94c60a11090e01b032d57/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:21.337540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.343801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '121487409c83478fa56d5c4ef9b87992'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.403026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce85499-62e6-4918-8646-f0e45127b1b1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.408080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.410475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.410475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.410475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.410475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.411445 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.422463 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1361/2720] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:21.426537 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.436651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35614373-c764-4795-b4cd-d88bed168336 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.441490 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a93bb9-31a5-4e19-a8c0-c4e1d3e98246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.442144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93bb9-31a5-4e19-a8c0-c4e1d3e98246 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.442452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93bb9-31a5-4e19-a8c0-c4e1d3e98246 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.442685 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a93bb9-31a5-4e19-a8c0-c4e1d3e98246 admin admin] PATH_INFO: `/v3/projects/d1a3b92f544949d8a813d902fdf212d9/users/611cd33d05c94c60a11090e01b032d57/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.444747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a93bb9-31a5-4e19-a8c0-c4e1d3e98246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a3b92f544949d8a813d902fdf212d9, user_id=611cd33d05c94c60a11090e01b032d57, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.447451 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a93bb9-31a5-4e19-a8c0-c4e1d3e98246 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.459094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1362/2721] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:21 2022] PUT /identity/v3/projects/d1a3b92f544949d8a813d902fdf212d9/users/611cd33d05c94c60a11090e01b032d57/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:21.473567 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.473850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.474130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.486285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '121487409c83478fa56d5c4ef9b87992'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.487160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '121487409c83478fa56d5c4ef9b87992'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.503827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '121487409c83478fa56d5c4ef9b87992'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.506620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] MFA Rules not processed for user `611cd33d05c94c60a11090e01b032d57`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:21.510006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:21.522116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '121487409c83478fa56d5c4ef9b87992'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:21.522116 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1360/2722] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:21 2022] DELETE /identity/v3/users/121487409c83478fa56d5c4ef9b87992 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:21.590137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1363/2723] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:21.598011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:21.599545 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:21.666067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ea79-d7a0-44f7-b4db-88436105db90 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:21.671476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:21.671476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:21.671476 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:21.672260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:21.674207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:21.741497 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:21.743042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1a74b-8f86-4ca0-aed1-32fafcb877dd service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:21.752930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1361/2724] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:22.034950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:22.038705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:22.112775 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804f942-630c-4bd9-9f41-3812c32bb0b3 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:22.139676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:22.172964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:22.173599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:22.173879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:22.174160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] PATH_INFO: `/v3/projects/8bbaf9777bb44dec89975bf98436ee92` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:22.175314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bbaf9777bb44dec89975bf98436ee92)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:22.177250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:22.336701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bbaf9777bb44dec89975bf98436ee92', 'request_id': 'req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:22.448520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bbaf9777bb44dec89975bf98436ee92', 'request_id': 'req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:22.449178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:22.455625 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bbaf9777bb44dec89975bf98436ee92', 'request_id': 'req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:22.468584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] The token cache is being invalidate because project 8bbaf9777bb44dec89975bf98436ee92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:22.468874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:22.469565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-cc14b93a-aa24-4777-b71c-e4f8e46bfe9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:22.471244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1364/2725] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:22 2022] DELETE /identity/v3/projects/8bbaf9777bb44dec89975bf98436ee92 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:22.476480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:22.477067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:22.481569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:22.499836 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:22.501093 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:22.581343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1362/2726] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:22.583422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:22.583776 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:22.584218 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:22.607702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:22.609357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:22.624641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1365/2727] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:22.625570 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:22.645151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ba531-c34d-4aa4-bf34-43f0e6bc9c51 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:22.651878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:22.652558 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:22.652980 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:22.653251 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:22.653903 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:22.654202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:22.655808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:22.667093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:22.681236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1363/2728] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:43:22.683833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:22.694338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f250d2-4d07-476c-b91c-0406a9deea8b None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:22.700016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:22.700661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:22.700942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:22.701210 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:22.701895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:22.703857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:22.837900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1366/2729] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:22 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:22.848955 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:22.860375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c1816-c942-4f33-8e1c-f026863feb9d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:22.920870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:22.921487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:22.921785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:22.922053 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:22.922713 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:22.924682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.033539 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1364/2730] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:22 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:23.034061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.045111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620f316-886b-4fd3-8276-0b310f81ad8d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.049870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.050464 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.050689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.050950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.051586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.053199 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.065749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1367/2731] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:23.070791 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.092061 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c35cdf-c32a-4c9c-8f02-7602dabd4862 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.099138 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e96e2-dd29-4cf3-a753-c45946d0bf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.099737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e96e2-dd29-4cf3-a753-c45946d0bf62 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.100019 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e96e2-dd29-4cf3-a753-c45946d0bf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.100291 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e96e2-dd29-4cf3-a753-c45946d0bf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.101211 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e96e2-dd29-4cf3-a753-c45946d0bf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.102823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e96e2-dd29-4cf3-a753-c45946d0bf62 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.114636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1365/2732] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:23.118971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.129502 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe68219-09da-465d-9d51-590d9cad83c9 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.134538 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.135262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.135571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.135682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:23.135842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] PATH_INFO: `/v3/projects/083cf7e77a1e436c88083f8b717d6425/users/bd01a71c592c4800a1c038fb70f99718/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.151407 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083cf7e77a1e436c88083f8b717d6425, user_id=bd01a71c592c4800a1c038fb70f99718, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.154090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.228501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:23.235770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1368/2733] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:23 2022] PUT /identity/v3/projects/083cf7e77a1e436c88083f8b717d6425/users/bd01a71c592c4800a1c038fb70f99718/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:23.247026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.247311 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.247605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.277459 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] MFA Rules not processed for user `bd01a71c592c4800a1c038fb70f99718`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:23.278711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:23.349381 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1366/2734] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:23.353904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.418661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a857-12a4-4494-9cbc-08dc46eb7f34 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.424299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.424919 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.425191 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.425456 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.426091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.427969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.522632 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1369/2735] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:23 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:23.528052 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.539451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156d3ac-1cb2-4d01-881f-d07ce330585b None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.606355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.606978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.607228 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.607438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.608066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.610611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.729385 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1367/2736] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:23 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:23.730868 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.748101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b0657-dcee-4c60-95c5-20dc279388a1 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.754278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:23.756795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.769518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.769518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.769518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.769518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.769518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.778700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1370/2737] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:23.783712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.799822 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348c119-1666-405f-91fb-83fdb1770955 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:23.807966 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:23.808943 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:23.809404 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:23.809677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] PATH_INFO: `/v3/projects/7ee635dd962545f59966df442b4ea451/users/406ec03539f34bf6812d23a4484a871d/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:23.831222 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee635dd962545f59966df442b4ea451, user_id=406ec03539f34bf6812d23a4484a871d, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:23.833999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:23.867013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:23.918111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1368/2738] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:23 2022] PUT /identity/v3/projects/7ee635dd962545f59966df442b4ea451/users/406ec03539f34bf6812d23a4484a871d/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:23.919684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:23.930953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005399f-b46b-4802-91d6-55c8c6b241bf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:24.008295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:24.008996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.009314 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.009759 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.011026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:24.012684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:24.026164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1371/2739] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:24.033490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:24.044237 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68dc08-3247-4f63-8398-3b1afdb8044d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:24.049842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5e250-27b5-4575-843e-62e68de8cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:24.050649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5e250-27b5-4575-843e-62e68de8cfeb admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.050913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5e250-27b5-4575-843e-62e68de8cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.051334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5e250-27b5-4575-843e-62e68de8cfeb admin admin] PATH_INFO: `/v3/projects/7ee635dd962545f59966df442b4ea451/users/406ec03539f34bf6812d23a4484a871d/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.058555 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5e250-27b5-4575-843e-62e68de8cfeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee635dd962545f59966df442b4ea451, user_id=406ec03539f34bf6812d23a4484a871d, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:24.059052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:24.061400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:24.061718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5e250-27b5-4575-843e-62e68de8cfeb admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:24.074010 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1369/2740] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:24 2022] PUT /identity/v3/projects/7ee635dd962545f59966df442b4ea451/users/406ec03539f34bf6812d23a4484a871d/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:24.076630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b2df-538a-4cba-b856-5617f0dd478f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:24.085894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.086181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.086451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.128871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] MFA Rules not processed for user `406ec03539f34bf6812d23a4484a871d`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:24.130048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:24.161985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:24.163256 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.163966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.164533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] PATH_INFO: `/v3/users/b3e018fe263643b4960506aed7343006` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.166400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e018fe263643b4960506aed7343006)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:24.170052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:24.233258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1370/2741] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:24.247973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:24.249538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:24.328349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef45f89-db69-41fc-82ff-a08fd1b9f2f4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:24.329122 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fc501-e0db-489d-b46a-99c7af1bd052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:24.329840 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fc501-e0db-489d-b46a-99c7af1bd052 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.330117 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fc501-e0db-489d-b46a-99c7af1bd052 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.330395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fc501-e0db-489d-b46a-99c7af1bd052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.343776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fc501-e0db-489d-b46a-99c7af1bd052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:24.345657 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fc501-e0db-489d-b46a-99c7af1bd052 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:24.358357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:24.358768 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1371/2742] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:24.418427 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e018fe263643b4960506aed7343006'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:24.431113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e018fe263643b4960506aed7343006'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:24.438840 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e018fe263643b4960506aed7343006'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:24.449408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:24.581098 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e018fe263643b4960506aed7343006'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:24.581795 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e018fe263643b4960506aed7343006'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:24.592994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1372/2743] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:24 2022] DELETE /identity/v3/users/b3e018fe263643b4960506aed7343006 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:24.597864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.598929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.598929 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.622360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:24.623602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:24.716894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1372/2744] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:24.731771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:24.733924 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:24.805850 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c4974-df93-4f57-9cff-0528541d756b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:24.809953 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:24.810562 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:24.810870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:24.811108 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:24.821960 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:24.824083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:24.834474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1373/2745] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:25.060428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:25.061968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:25.075278 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f2f5f-2dac-4ffe-94d6-c6d9ed69fe6e None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:25.080250 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:25.080977 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:25.081209 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:25.081480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] PATH_INFO: `/v3/users/bd01a71c592c4800a1c038fb70f99718` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:25.082664 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd01a71c592c4800a1c038fb70f99718)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:25.084726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:25.293753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd01a71c592c4800a1c038fb70f99718'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.344783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:25.346881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:25.363966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f8d89-7136-4148-9a50-2599ed63f58a service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:25.369061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:25.369975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:25.370201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:25.370434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] PATH_INFO: `/v3/projects/0184528a6c274737a4a70e87465e825b` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:25.371728 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0184528a6c274737a4a70e87465e825b)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:25.373792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:25.410096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd01a71c592c4800a1c038fb70f99718'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.411253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd01a71c592c4800a1c038fb70f99718'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.431501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:25.433851 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd01a71c592c4800a1c038fb70f99718'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.441074 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c13d7eb1-33f2-49fc-b663-cb0f5c8036e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd01a71c592c4800a1c038fb70f99718'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.447972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1373/2746] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:25 2022] DELETE /identity/v3/users/bd01a71c592c4800a1c038fb70f99718 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:25.452457 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:25.452773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:25.456892 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:25.477833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:25.480120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:25.542004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1374/2747] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:25.560049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:25.563438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:25.592975 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0184528a6c274737a4a70e87465e825b', 'request_id': 'req-e42390f0-c4aa-49ad-b24b-e4e9c0582217'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.637273 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642881b-fa58-44fc-a728-620291bcefaf None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:25.711834 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:25.712630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:25.712969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:25.713374 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:25.788244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:25.789773 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:25.800235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1375/2748] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:25.827194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0184528a6c274737a4a70e87465e825b', 'request_id': 'req-e42390f0-c4aa-49ad-b24b-e4e9c0582217'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.835649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0184528a6c274737a4a70e87465e825b', 'request_id': 'req-e42390f0-c4aa-49ad-b24b-e4e9c0582217'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.850315 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] The token cache is being invalidate because project 0184528a6c274737a4a70e87465e825b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:25.850790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.851649 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:25.853954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1374/2749] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:25 2022] DELETE /identity/v3/projects/0184528a6c274737a4a70e87465e825b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:25.858538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:25.860438 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:25.876102 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf9de-55af-4cb7-939d-d40fe10d5bad service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:25.882863 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:25.883736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:25.884127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:25.884521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] PATH_INFO: `/v3/users/8d8ef841ac63474aa89e234231b91edf` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:25.887011 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8ef841ac63474aa89e234231b91edf)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:25.890071 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:26.162603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8ef841ac63474aa89e234231b91edf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.301130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8ef841ac63474aa89e234231b91edf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.304665 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8ef841ac63474aa89e234231b91edf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.315243 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8ef841ac63474aa89e234231b91edf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.322098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8ef841ac63474aa89e234231b91edf'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.328181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1376/2750] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:25 2022] DELETE /identity/v3/users/8d8ef841ac63474aa89e234231b91edf => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:26.378613 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:26.380092 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:26.393985 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42390f0-c4aa-49ad-b24b-e4e9c0582217 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:26.472158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:26.472914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:26.473211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:26.473458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] PATH_INFO: `/v3/projects/083cf7e77a1e436c88083f8b717d6425` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:26.474589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=083cf7e77a1e436c88083f8b717d6425)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:26.476533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:26.484514 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:26.671220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083cf7e77a1e436c88083f8b717d6425', 'request_id': 'req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.821143 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083cf7e77a1e436c88083f8b717d6425', 'request_id': 'req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.828797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083cf7e77a1e436c88083f8b717d6425', 'request_id': 'req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.841781 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] The token cache is being invalidate because project 083cf7e77a1e436c88083f8b717d6425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:26.842088 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.842731 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:26.844236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1375/2751] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:26 2022] DELETE /identity/v3/projects/083cf7e77a1e436c88083f8b717d6425 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:26.848396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:26.849873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:26.863900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61299c26-d85d-4729-9342-970e5a935a6d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:26.919684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:26.922227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:26.927609 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:26.928263 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:26.929186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:26.929186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] PATH_INFO: `/v3/users/406ec03539f34bf6812d23a4484a871d` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:26.930364 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=406ec03539f34bf6812d23a4484a871d)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:26.933896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:26.937404 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da549cbd-67ab-44d8-81ef-3ea4ff1364f8 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:26.949395 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:26.949936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:26.950280 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:26.950475 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] PATH_INFO: `/v3/projects/a2fb93b6f8554f9cbc9c22400d86f6c7` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:26.951869 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2fb93b6f8554f9cbc9c22400d86f6c7)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:26.954016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.165547 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406ec03539f34bf6812d23a4484a871d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.167615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2fb93b6f8554f9cbc9c22400d86f6c7', 'request_id': 'req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.249059 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2fb93b6f8554f9cbc9c22400d86f6c7', 'request_id': 'req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.255430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2fb93b6f8554f9cbc9c22400d86f6c7', 'request_id': 'req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.267856 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] The token cache is being invalidate because project a2fb93b6f8554f9cbc9c22400d86f6c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:27.268144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.268846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f4ee6bf9-d7b4-4216-b6cd-bfcbebe64385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.270535 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1376/2752] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:26 2022] DELETE /identity/v3/projects/a2fb93b6f8554f9cbc9c22400d86f6c7 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:27.276596 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727f101-be30-49ef-a97f-c4074965257a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.277058 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727f101-be30-49ef-a97f-c4074965257a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.277231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727f101-be30-49ef-a97f-c4074965257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.298583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d727f101-be30-49ef-a97f-c4074965257a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:27.299814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d727f101-be30-49ef-a97f-c4074965257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:27.306305 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406ec03539f34bf6812d23a4484a871d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.311700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406ec03539f34bf6812d23a4484a871d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.325493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406ec03539f34bf6812d23a4484a871d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.332717 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-b36f04c9-a641-4a89-9e0b-2eb07a578885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406ec03539f34bf6812d23a4484a871d'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:27.340932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1377/2753] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:26 2022] DELETE /identity/v3/users/406ec03539f34bf6812d23a4484a871d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:27.383756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1377/2754] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:27.386056 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.386336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.386603 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.409433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:27.410853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:27.483479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1378/2755] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:27.488200 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727f101-be30-49ef-a97f-c4074965257a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:27.498226 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727f101-be30-49ef-a97f-c4074965257a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:27.503224 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:27.503789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.504064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.504349 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.505094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:27.505391 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:27.506983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.520233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1378/2756] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:43:27.527598 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:27.537479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dbb70-a1fe-4d1f-9489-549d7c48a023 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:27.542392 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:27.543207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.544750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.546823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.546823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:27.547736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.686652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1379/2757] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:27 2022] POST /identity/v3/projects => generated 389 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:27.690958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:27.707804 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dbc1a-469c-4d3e-bfd4-9a03a5e4f820 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:27.725783 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:27.727629 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:27.803091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:27.803719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.804045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.804362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.804947 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:27.806911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.807627 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e258-4159-4964-a260-475065ec28fd admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:27.814758 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:27.815735 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.816032 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.816562 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.820652 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:27.830942 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:27.906286 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:27.907065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.919216 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1380/2758] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:27.920269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:27.921692 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:27.925260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1379/2759] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:27 2022] POST /identity/v3/users => generated 295 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:27.933360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:27.936172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70857d-caa7-4b7b-b3f8-f81f6bc9dd51 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:27.947096 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:27.952501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.952501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.952501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] PATH_INFO: `/v3/projects/7ee635dd962545f59966df442b4ea451` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.952501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee635dd962545f59966df442b4ea451)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:27.952501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.956281 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690bf66-8c81-46a5-ab45-24ab22fae5a3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:27.966263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:27.967135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:27.967523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:27.968215 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:27.969211 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:27.971583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:27.987910 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1380/2760] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:27.992408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.002719 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a63783-73bd-4afa-b803-2db7f07680db admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.007594 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.008157 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.008478 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.008749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.009422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.011811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.022758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1381/2761] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:28.028590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.038948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30fce0-aaa9-437a-ad7e-308b36d3359e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.044819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe962230-6956-48c8-8c58-3016e0c4ce56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.045500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe962230-6956-48c8-8c58-3016e0c4ce56 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.045760 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe962230-6956-48c8-8c58-3016e0c4ce56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.046033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe962230-6956-48c8-8c58-3016e0c4ce56 admin admin] PATH_INFO: `/v3/projects/1d2f2b8122874d15aa5430491bd441a7/users/578402e476a54817b034b3c63e1ee88f/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.061702 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe962230-6956-48c8-8c58-3016e0c4ce56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2f2b8122874d15aa5430491bd441a7, user_id=578402e476a54817b034b3c63e1ee88f, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.064458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe962230-6956-48c8-8c58-3016e0c4ce56 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.216861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1382/2762] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:28 2022] PUT /identity/v3/projects/1d2f2b8122874d15aa5430491bd441a7/users/578402e476a54817b034b3c63e1ee88f/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:28.224733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee635dd962545f59966df442b4ea451', 'request_id': 'req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:28.227399 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.227746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.228026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.230944 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee635dd962545f59966df442b4ea451', 'request_id': 'req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:28.237272 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee635dd962545f59966df442b4ea451', 'request_id': 'req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:28.258364 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] MFA Rules not processed for user `578402e476a54817b034b3c63e1ee88f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:28.261689 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:28.333598 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1383/2763] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:28.337571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] The token cache is being invalidate because project 7ee635dd962545f59966df442b4ea451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:28.337571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:28.337571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-fcba8393-d4d4-4a38-a865-f1ae8531c4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:28.342238 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.342354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1381/2764] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:27 2022] DELETE /identity/v3/projects/7ee635dd962545f59966df442b4ea451 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:28.346144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143f370-b7d2-4e2e-bf1f-06a1599d0e3a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.346749 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143f370-b7d2-4e2e-bf1f-06a1599d0e3a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.347379 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143f370-b7d2-4e2e-bf1f-06a1599d0e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.377471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e143f370-b7d2-4e2e-bf1f-06a1599d0e3a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:28.378871 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e143f370-b7d2-4e2e-bf1f-06a1599d0e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:28.405992 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba5ac-6210-4c75-be8f-2ff3503bca5f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.410825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.411450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.411800 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.411991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.412704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.414736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.445630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1382/2765] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:28.458591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.458868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.459159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.590079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1384/2766] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/projects => generated 389 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:28.595054 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.596930 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:28.607536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:28.608839 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:28.669512 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826cfe5-6c80-42f4-9460-8b567b07f729 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.674697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.675317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.675692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.675997 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.676679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.678705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.680868 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1383/2767] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:28.686366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.695636 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b03b-f361-4739-a01a-b096fefe5de5 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.700753 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.701358 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.701656 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.701911 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.702573 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:28.702859 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.704721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.717240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1384/2768] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:43:28.721601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.732159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acea910-bd29-4026-b731-cf342894e355 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.737297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.737890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.738169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.738440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.739134 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.741078 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.795884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1385/2769] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/users => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:28.796999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.808573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42e046-6c1a-40f6-a49a-ee9127340be4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.814093 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.814721 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.814994 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.815258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.815886 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.817750 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.830860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1386/2770] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:28.835583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.846006 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b664d-ec01-4ac1-a6e2-26ac0e1eea87 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.850970 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.851615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.851887 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.852187 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] PATH_INFO: `/v3/projects/a6faa2cb1ae34775ab5431eda54a76af/users/cbffe283fa75437b8e949afb242479a6/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.868311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6faa2cb1ae34775ab5431eda54a76af, user_id=cbffe283fa75437b8e949afb242479a6, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.871797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.884575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1385/2771] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:28.886481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:28.952843 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f962-2874-486f-a7b6-03745955aaee admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:28.957985 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:28.958655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:28.963420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:28.963420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:28.963420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:28.963420 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:28.997893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1387/2772] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:28 2022] PUT /identity/v3/projects/a6faa2cb1ae34775ab5431eda54a76af/users/cbffe283fa75437b8e949afb242479a6/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:29.001704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.069879 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325575-b1b2-4e3e-8e56-201a1787ba18 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.075342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.075454 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.075749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.076001 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.076671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.078251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.089849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1388/2773] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:29.094510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1386/2774] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:28 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:29.098666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.109669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.112943 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b05d7-09b8-4e4b-a1c4-9c9ea846ad14 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.114375 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c10bb-b13f-4027-bdd6-a8b6299e731a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.118690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089063fd-2860-4d74-856b-54568c6216a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.119220 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089063fd-2860-4d74-856b-54568c6216a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.119491 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089063fd-2860-4d74-856b-54568c6216a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.121032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089063fd-2860-4d74-856b-54568c6216a6 admin admin] PATH_INFO: `/v3/projects/a6faa2cb1ae34775ab5431eda54a76af/users/cbffe283fa75437b8e949afb242479a6/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.122423 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.123142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.123554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.123924 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.124234 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089063fd-2860-4d74-856b-54568c6216a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6faa2cb1ae34775ab5431eda54a76af, user_id=cbffe283fa75437b8e949afb242479a6, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.124826 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.126679 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.127360 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089063fd-2860-4d74-856b-54568c6216a6 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.145424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1387/2776] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:29.152485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.157027 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1389/2776] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:29 2022] PUT /identity/v3/projects/a6faa2cb1ae34775ab5431eda54a76af/users/cbffe283fa75437b8e949afb242479a6/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:29.169979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831694b-5b30-4660-a258-07134fee87a2 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.213010 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.213329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.213602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.240832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.241443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.241725 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.242046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.242693 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.244767 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.253927 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] MFA Rules not processed for user `cbffe283fa75437b8e949afb242479a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:29.255139 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:29.260876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1388/2777] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:29.268762 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.280979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2128-442f-4379-acde-5f6b5b8d55cb admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.286677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8013081-8cf5-4ce3-8a6d-6d25a9b27b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.287716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8013081-8cf5-4ce3-8a6d-6d25a9b27b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.288141 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8013081-8cf5-4ce3-8a6d-6d25a9b27b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.288411 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8013081-8cf5-4ce3-8a6d-6d25a9b27b20 admin admin] PATH_INFO: `/v3/projects/423943e4de9b4b389cc6d38a10a525ed/users/224f1253e6f24c4f9bd351657c65160e/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.306288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8013081-8cf5-4ce3-8a6d-6d25a9b27b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423943e4de9b4b389cc6d38a10a525ed, user_id=224f1253e6f24c4f9bd351657c65160e, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.309202 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8013081-8cf5-4ce3-8a6d-6d25a9b27b20 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.333362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1390/2778] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:29.344703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.346096 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:29.403249 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1389/2779] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:29 2022] PUT /identity/v3/projects/423943e4de9b4b389cc6d38a10a525ed/users/224f1253e6f24c4f9bd351657c65160e/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:29.407800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.407800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.407800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.413248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489963f-1b4c-47f2-ab2d-aad6ab78898a None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.440430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] MFA Rules not processed for user `224f1253e6f24c4f9bd351657c65160e`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:29.442352 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:29.461787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:29.478578 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.480622 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.481113 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.482063 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.522043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1390/2780] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:29.534697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:29.536092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:29.555523 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.557078 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.567861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1391/2781] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:29.597037 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bfc3b-61ad-486f-86bf-8fce06447036 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:29.601511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21a7e9-9b63-4531-aae9-58043d7e2d8a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:29.602129 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21a7e9-9b63-4531-aae9-58043d7e2d8a service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:29.602407 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21a7e9-9b63-4531-aae9-58043d7e2d8a service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:29.602682 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21a7e9-9b63-4531-aae9-58043d7e2d8a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:29.612168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21a7e9-9b63-4531-aae9-58043d7e2d8a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:29.613705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21a7e9-9b63-4531-aae9-58043d7e2d8a service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:29.623113 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1391/2782] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:30.072433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:30.291369 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:30.375452 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee53d4-d29f-4ad0-acd9-2a8d82b9e7e5 service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:30.380213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:30.380964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:30.381287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:30.381550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] PATH_INFO: `/v3/users/578402e476a54817b034b3c63e1ee88f` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:30.382671 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=578402e476a54817b034b3c63e1ee88f)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:30.384660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:30.667923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578402e476a54817b034b3c63e1ee88f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:30.679788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578402e476a54817b034b3c63e1ee88f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:30.687295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578402e476a54817b034b3c63e1ee88f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:30.792772 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578402e476a54817b034b3c63e1ee88f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:30.793443 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578402e476a54817b034b3c63e1ee88f'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:30.800268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1392/2783] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:30 2022] DELETE /identity/v3/users/578402e476a54817b034b3c63e1ee88f => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:30.804514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:30.808275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:30.808275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:30.824369 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:30.825596 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:30.896933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1392/2784] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:30.908401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:30.910012 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:31.007032 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc2185-7e01-4efa-92e8-ab68ceb52a2f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:31.012692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:31.013511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:31.013787 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:31.014181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:31.025626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:31.042921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:31.042921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1393/2785] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:31.582428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:31.584018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:31.601050 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8569c-b0bd-4320-90d3-2a76aa915916 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:31.609395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:31.610493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:31.610982 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:31.611490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] PATH_INFO: `/v3/projects/1d2f2b8122874d15aa5430491bd441a7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:31.613258 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2f2b8122874d15aa5430491bd441a7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:31.616118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:31.758263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:31.783450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2f2b8122874d15aa5430491bd441a7', 'request_id': 'req-75e297a1-ba49-4402-a30a-d16f5de5f0d0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:31.789618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2f2b8122874d15aa5430491bd441a7', 'request_id': 'req-75e297a1-ba49-4402-a30a-d16f5de5f0d0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:31.797472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2f2b8122874d15aa5430491bd441a7', 'request_id': 'req-75e297a1-ba49-4402-a30a-d16f5de5f0d0'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:31.871801 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] The token cache is being invalidate because project 1d2f2b8122874d15aa5430491bd441a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:31.872062 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:31.872779 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:31.874491 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1393/2786] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:31 2022] DELETE /identity/v3/projects/1d2f2b8122874d15aa5430491bd441a7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:31.881785 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:31.899441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b05bc-10b0-460c-afee-07b63fea4843 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:31.973792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:31.974254 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:31.974559 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:31.974853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] PATH_INFO: `/v3/users/cbffe283fa75437b8e949afb242479a6` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:31.975861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbffe283fa75437b8e949afb242479a6)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:31.977822 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:31.998017 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:32.202855 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbffe283fa75437b8e949afb242479a6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:32.216898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbffe283fa75437b8e949afb242479a6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:32.226818 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbffe283fa75437b8e949afb242479a6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:32.283860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:32.312068 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbffe283fa75437b8e949afb242479a6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:32.312734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-138a1531-467b-4e4e-9c74-7660e11d8f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbffe283fa75437b8e949afb242479a6'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:32.319705 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1394/2787] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:31 2022] DELETE /identity/v3/users/cbffe283fa75437b8e949afb242479a6 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:32.489856 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:32.609959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=100 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:32.788947 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:32.790740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:32.810989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e297a1-ba49-4402-a30a-d16f5de5f0d0 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:32.855789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:32.871067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:32.871711 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:32.871988 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:32.872252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] PATH_INFO: `/v3/projects/a6faa2cb1ae34775ab5431eda54a76af` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:32.877625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6faa2cb1ae34775ab5431eda54a76af)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:32.877625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.002164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6faa2cb1ae34775ab5431eda54a76af', 'request_id': 'req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:33.008178 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6faa2cb1ae34775ab5431eda54a76af', 'request_id': 'req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:33.015247 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6faa2cb1ae34775ab5431eda54a76af', 'request_id': 'req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:33.105876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] The token cache is being invalidate because project a6faa2cb1ae34775ab5431eda54a76af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:33.106588 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:33.107313 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-0cc4baa0-e3ba-4b2c-85e4-01766c78bd49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:33.109079 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1394/2788] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:32 2022] DELETE /identity/v3/projects/a6faa2cb1ae34775ab5431eda54a76af => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:33.115276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.115583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.115853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.137554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:33.138967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:33.211492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1395/2789] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:33.225306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.226120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.226702 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.232684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:33.252730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:33.253878 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:33.261855 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1395/2790] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:33.266321 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:33.276771 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e69cfd-d8fc-4617-bc02-ec5c9dc047a6 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:33.281593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:33.283194 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.283477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.283752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.284453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:33.284768 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:33.286352 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.299271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1396/2791] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:33.304611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:33.316033 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc6cfc-734e-4845-943d-dfd515ceaff6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:33.321885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:33.322501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.327253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.327253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.327253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:33.327253 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.439174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1396/2792] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:33 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:33.446440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:33.459252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3aa6b-224a-40a9-8f57-58eba626c5d6 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:33.473057 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:33.496592 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:33.537430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:33.537987 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.538267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.538542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.539179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:33.541163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.690069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1397/2793] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:33 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:33.694362 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:33.705622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7fac89-146f-4861-95fc-86d94db3df90 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:33.710287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:33.710879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.711151 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.711554 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.712186 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:33.713825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.724723 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1397/2794] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:33.729174 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:33.739969 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb023a-9305-4a68-8a37-2910a0d3f06f admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:33.744877 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9ffed-b8b4-40f9-b294-0cde27f90dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:33.746003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9ffed-b8b4-40f9-b294-0cde27f90dce admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.746285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9ffed-b8b4-40f9-b294-0cde27f90dce admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.746548 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9ffed-b8b4-40f9-b294-0cde27f90dce admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.747185 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9ffed-b8b4-40f9-b294-0cde27f90dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:33.748811 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9ffed-b8b4-40f9-b294-0cde27f90dce admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.759746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1398/2795] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:33.764744 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:33.780833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3528c08-e6a8-47f8-9599-9c82e484b26a admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:33.780833 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:33.781085 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.781366 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.781640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] PATH_INFO: `/v3/projects/58846ca5da5e4829a4166f9d7587c9f0/users/6aac9badb0d44823b91c38a119ac3924/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.798904 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=58846ca5da5e4829a4166f9d7587c9f0, user_id=6aac9badb0d44823b91c38a119ac3924, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:33.801630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:33.953308 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1398/2796] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:33 2022] PUT /identity/v3/projects/58846ca5da5e4829a4166f9d7587c9f0/users/6aac9badb0d44823b91c38a119ac3924/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:33.959876 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:33.960701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:33.961101 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:33.982788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:33.984033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:34.050184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1399/2797] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:34.061171 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:34.062941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:34.099865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:34.123946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703ba7-9dbd-4552-a17a-3c4b20dc9b2e admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:34.128055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6685b97-8a8c-4b1a-9c36-d32c7f3869e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:34.128736 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6685b97-8a8c-4b1a-9c36-d32c7f3869e6 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:34.129016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6685b97-8a8c-4b1a-9c36-d32c7f3869e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:34.129287 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6685b97-8a8c-4b1a-9c36-d32c7f3869e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:34.140260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6685b97-8a8c-4b1a-9c36-d32c7f3869e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:34.141740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6685b97-8a8c-4b1a-9c36-d32c7f3869e6 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:34.151999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1399/2798] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:34.686829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:34.688633 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:34.702097 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8cc8d-79cd-4d8b-8126-aa0aba0b6e10 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:34.707110 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:34.707817 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:34.708163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:34.708479 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] PATH_INFO: `/v3/users/a3e774303f804a818739ad09b7bfc73c` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:34.709610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e774303f804a818739ad09b7bfc73c)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:34.711575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:35.039030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e774303f804a818739ad09b7bfc73c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:35.058620 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e774303f804a818739ad09b7bfc73c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:35.067846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e774303f804a818739ad09b7bfc73c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:35.152336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e774303f804a818739ad09b7bfc73c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:35.153075 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e774303f804a818739ad09b7bfc73c'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:35.160083 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1400/2799] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:34 2022] DELETE /identity/v3/users/a3e774303f804a818739ad09b7bfc73c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:35.164671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a058b5-0d36-4e6e-910c-e6a89269bbce None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:35.165196 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a058b5-0d36-4e6e-910c-e6a89269bbce None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:35.165294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a058b5-0d36-4e6e-910c-e6a89269bbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:35.186306 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-34a058b5-0d36-4e6e-910c-e6a89269bbce None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:35.187511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-34a058b5-0d36-4e6e-910c-e6a89269bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:35.265227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1400/2800] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:43:35.275309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:35.277692 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:35.295888 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:35.347698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e50edd-33bb-4ac3-b01d-459b3dc09c79 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:35.351684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:35.352434 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:35.352734 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:35.353023 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:35.366351 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:35.367833 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:35.380271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1401/2801] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:35.646983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:35.647345 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:35.647585 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:35.685095 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] MFA Rules not processed for user `6aac9badb0d44823b91c38a119ac3924`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:35.686660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:35.785870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1401/2802] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:35.790923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:35.793401 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:35.806928 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bf3e7-6b77-4f63-aa13-08f7c3a33389 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:35.813049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:35.813049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:35.813336 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:35.813607 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:35.814239 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:35.816168 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:35.907782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:35.910451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:35.934048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1402/2803] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:35 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:35.937894 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:35.942639 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b6abf-70f0-4b16-8373-c10ae7d706c4 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.015149 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.015790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.016072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.016487 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] PATH_INFO: `/v3/projects/8281363ad1c64e6393af42793f9d55c7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.017544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8281363ad1c64e6393af42793f9d55c7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.019628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.029479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:36.031018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c470b-004d-467c-a1cf-fd405246242d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.036038 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.036610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.036889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.037159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.037788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.039907 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.148648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8281363ad1c64e6393af42793f9d55c7', 'request_id': 'req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.155200 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8281363ad1c64e6393af42793f9d55c7', 'request_id': 'req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.161888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8281363ad1c64e6393af42793f9d55c7', 'request_id': 'req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.164981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1403/2804] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:35 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:36.170050 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:36.190380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62594c-4a0e-4143-b57b-c920fccff116 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.243828 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] The token cache is being invalidate because project 8281363ad1c64e6393af42793f9d55c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:36.244716 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.245224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.246787 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1402/2805] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:35 2022] DELETE /identity/v3/projects/8281363ad1c64e6393af42793f9d55c7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:36.250910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:36.252399 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:36.264971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.265615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.265892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.266291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.266949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.268996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.283382 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1404/2806] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:36.289048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:36.291052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:36.306249 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d94b3-be36-4aa6-8c4e-255d7de0ecbf admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.312453 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.313148 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.313433 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.313720 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] PATH_INFO: `/v3/projects/66456a796af547c190c9d32a2603bcdc/users/ac82aed584af4b0a815bdd43e83f8fbf/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.331597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66456a796af547c190c9d32a2603bcdc, user_id=ac82aed584af4b0a815bdd43e83f8fbf, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.334329 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.336756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:36.337109 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5eb0f-c0a4-4d83-bac3-9fbdda162266 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.342575 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.343210 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.343517 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.343788 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] PATH_INFO: `/v3/users/611cd33d05c94c60a11090e01b032d57` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.344958 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=611cd33d05c94c60a11090e01b032d57)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.355132 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.442664 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1405/2807] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:36 2022] PUT /identity/v3/projects/66456a796af547c190c9d32a2603bcdc/users/ac82aed584af4b0a815bdd43e83f8fbf/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:36.455132 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:36.522130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a98e74-d4b6-4062-9ac6-75a005cd6b18 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.528554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.528554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.538636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.538636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.538636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.538636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.543150 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1406/2808] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:36.549998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:36.564166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1695d4-fc19-43b4-9737-116e45b319cb admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:36.569569 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:36.578488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:36.578488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:36.578488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] PATH_INFO: `/v3/projects/66456a796af547c190c9d32a2603bcdc/users/ac82aed584af4b0a815bdd43e83f8fbf/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:36.578488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66456a796af547c190c9d32a2603bcdc, user_id=ac82aed584af4b0a815bdd43e83f8fbf, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:36.578488 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:36.587064 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1407/2809] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:36 2022] PUT /identity/v3/projects/66456a796af547c190c9d32a2603bcdc/users/ac82aed584af4b0a815bdd43e83f8fbf/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:43:36.625790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '611cd33d05c94c60a11090e01b032d57'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.712144 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '611cd33d05c94c60a11090e01b032d57'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.712842 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '611cd33d05c94c60a11090e01b032d57'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.729405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '611cd33d05c94c60a11090e01b032d57'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.738894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-ea2249cf-e984-4f42-a1b1-87652e98b0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '611cd33d05c94c60a11090e01b032d57'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:36.744712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1403/2810] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:36 2022] DELETE /identity/v3/users/611cd33d05c94c60a11090e01b032d57 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:37.251137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:37.251137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:37.335998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ec1a-0d71-4570-ad1c-d0474939d0d5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:37.405111 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:37.405749 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:37.406026 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:37.406295 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] PATH_INFO: `/v3/projects/d1a3b92f544949d8a813d902fdf212d9` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:37.407477 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a3b92f544949d8a813d902fdf212d9)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:37.409455 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:37.555016 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:37.576674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a3b92f544949d8a813d902fdf212d9', 'request_id': 'req-f65e9223-e983-424f-ad21-f340cd113848'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:37.672498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a3b92f544949d8a813d902fdf212d9', 'request_id': 'req-f65e9223-e983-424f-ad21-f340cd113848'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:37.678949 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a3b92f544949d8a813d902fdf212d9', 'request_id': 'req-f65e9223-e983-424f-ad21-f340cd113848'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:37.692135 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] The token cache is being invalidate because project d1a3b92f544949d8a813d902fdf212d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:43:37.692662 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:37.693895 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f65e9223-e983-424f-ad21-f340cd113848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:37.701481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1408/2811] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:37 2022] DELETE /identity/v3/projects/d1a3b92f544949d8a813d902fdf212d9 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:37.711098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:37.711098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:37.711098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:37.739163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:37.739163 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:37.750709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:37.816016 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1404/2812] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:37.825542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:37.825846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:37.826166 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:37.854137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:37.855651 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:37.864045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1409/2813] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:37.868895 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:37.879466 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548b873-9e2e-4b4f-a763-6031ea946556 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:37.884510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:37.885112 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:37.885390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:37.886208 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:37.886894 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:43:37.887189 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:37.888862 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:37.902357 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1405/2814] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:43:37.907287 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:37.918440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fa7a9-5ff6-40af-abda-e2cef2fa5d8f None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:37.923714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:37.924342 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:37.924637 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:37.924904 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:37.925555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:37.927499 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:38.045813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1410/2815] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:43:37 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:43:38.050442 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:38.062110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b6e4-9cff-443d-91f5-99a564f12f65 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:38.125510 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:38.126233 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.126520 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.127115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.127825 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:38.130288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:38.272105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1406/2816] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:43:38 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:43:38.278672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:38.291762 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9364965-15b9-44c0-ae42-1500e1eadb78 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:38.297437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:38.298051 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.300801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.300801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.300801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:38.301278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:38.311913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1411/2817] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:43:38.316502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:38.327618 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6ad9c-6d57-4604-b206-0d7203799cc5 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:38.332888 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fd33d-177f-4a54-830c-fc79ecaad266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:38.333577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fd33d-177f-4a54-830c-fc79ecaad266 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.333854 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fd33d-177f-4a54-830c-fc79ecaad266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.334120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fd33d-177f-4a54-830c-fc79ecaad266 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.334857 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fd33d-177f-4a54-830c-fc79ecaad266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:38.336522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fd33d-177f-4a54-830c-fc79ecaad266 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:38.355465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1407/2818] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:43:38.361697 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:38.374813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3ae8-fdf8-46a1-af4a-398e73dbffb5 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:38.379815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf343a-d5f0-4f51-82a4-48e26941cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:38.380519 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf343a-d5f0-4f51-82a4-48e26941cbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.380842 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf343a-d5f0-4f51-82a4-48e26941cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.381152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf343a-d5f0-4f51-82a4-48e26941cbc7 admin admin] PATH_INFO: `/v3/projects/b41f7b96c8474f0f89b93473eb73d7ee/users/cbffc35b26af4f13bcb582d502f978f2/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.385471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.386347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.386347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.406881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf343a-d5f0-4f51-82a4-48e26941cbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41f7b96c8474f0f89b93473eb73d7ee, user_id=cbffc35b26af4f13bcb582d502f978f2, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:38.410437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf343a-d5f0-4f51-82a4-48e26941cbc7 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:38.431328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] MFA Rules not processed for user `ac82aed584af4b0a815bdd43e83f8fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:38.433174 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:38.485952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1412/2819] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:43:38 2022] PUT /identity/v3/projects/b41f7b96c8474f0f89b93473eb73d7ee/users/cbffc35b26af4f13bcb582d502f978f2/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:38.499717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.499999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.500267 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.514831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1408/2820] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:38.527640 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:38.529058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:38.535896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] MFA Rules not processed for user `cbffc35b26af4f13bcb582d502f978f2`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:43:38.537164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:38.539930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:38.606218 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c166d-f3ec-45f4-b817-30876e72c7f6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:38.622790 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1413/2821] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:43:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:43:38.666569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:38.667780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:38.668055 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:38.668435 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:38.670370 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:38.736443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:38.737880 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:38.747688 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1409/2822] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:40.464954 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:40.466419 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:40.478838 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5463f-5792-4ee1-acb2-69c975a4d83d None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:40.483069 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:40.485786 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:40.485892 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:40.486134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:40.527638 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:40.552086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:40.553679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:40.565674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1414/2823] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:43:48.520091 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:48.522910 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:48.588268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ab00f-e63f-4598-8b0e-69aa783c03d4 service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:48.592021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:48.592662 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:48.592941 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:48.593215 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:48.602440 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:48.604121 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:48.614696 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1410/2824] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:50.698317 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=12 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:58.609714 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=91 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:58.839669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:58.846474 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:43:58.923897 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969fd1e-7a4f-4912-97c1-4b6acd54707b service cinder] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:58.931039 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:58.931819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:58.932184 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:58.932603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] PATH_INFO: `/v3/users/6aac9badb0d44823b91c38a119ac3924` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:58.933958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aac9badb0d44823b91c38a119ac3924)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:58.938225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:59.332918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aac9badb0d44823b91c38a119ac3924'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.346995 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aac9badb0d44823b91c38a119ac3924'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.354309 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aac9badb0d44823b91c38a119ac3924'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.426672 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aac9badb0d44823b91c38a119ac3924'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.428217 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aac9badb0d44823b91c38a119ac3924'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.434932 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1415/2825] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:43:58 2022] DELETE /identity/v3/users/6aac9badb0d44823b91c38a119ac3924 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:43:59.715013 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:59.726303 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222009d-36d1-4bd2-8a14-71dff17f4899 service glance] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:59.789678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:59.790470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:59.790709 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:59.790983 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] PATH_INFO: `/v3/projects/58846ca5da5e4829a4166f9d7587c9f0` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:59.792747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58846ca5da5e4829a4166f9d7587c9f0)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:59.794820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:59.981740 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58846ca5da5e4829a4166f9d7587c9f0', 'request_id': 'req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.987741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58846ca5da5e4829a4166f9d7587c9f0', 'request_id': 'req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:43:59.993992 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58846ca5da5e4829a4166f9d7587c9f0', 'request_id': 'req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.117502 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] The token cache is being invalidate because project 58846ca5da5e4829a4166f9d7587c9f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:44:00.118059 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.118430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.120106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1411/2826] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:43:59 2022] DELETE /identity/v3/projects/58846ca5da5e4829a4166f9d7587c9f0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:00.124424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:00.126196 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:00.141232 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1ebe-0896-469b-ae90-10ab30d57f0e admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:00.208674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:00.209400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:00.209686 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:00.209950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] PATH_INFO: `/v3/users/ac82aed584af4b0a815bdd43e83f8fbf` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:00.211116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac82aed584af4b0a815bdd43e83f8fbf)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:00.213080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:00.246934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:00.479043 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac82aed584af4b0a815bdd43e83f8fbf'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.493067 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac82aed584af4b0a815bdd43e83f8fbf'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.500767 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac82aed584af4b0a815bdd43e83f8fbf'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.619025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac82aed584af4b0a815bdd43e83f8fbf'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.633711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac82aed584af4b0a815bdd43e83f8fbf'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:00.633711 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1416/2827] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:44:00 2022] DELETE /identity/v3/users/ac82aed584af4b0a815bdd43e83f8fbf => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:44:00.931583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:00.933207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:00.947965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef00c52-f2d3-4f7e-b8b7-b5f37a6c5a05 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:01.013051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:01.013611 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:01.013889 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:01.014167 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] PATH_INFO: `/v3/projects/66456a796af547c190c9d32a2603bcdc` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:01.016548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66456a796af547c190c9d32a2603bcdc)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:01.018512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:01.215386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:01.227066 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66456a796af547c190c9d32a2603bcdc', 'request_id': 'req-572b6893-c8c2-45ef-9083-d62987787a02'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:01.233776 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66456a796af547c190c9d32a2603bcdc', 'request_id': 'req-572b6893-c8c2-45ef-9083-d62987787a02'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:01.240297 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66456a796af547c190c9d32a2603bcdc', 'request_id': 'req-572b6893-c8c2-45ef-9083-d62987787a02'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:01.346084 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] The token cache is being invalidate because project 66456a796af547c190c9d32a2603bcdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:44:01.348428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:01.348428 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-572b6893-c8c2-45ef-9083-d62987787a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:01.350035 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1412/2828] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:44:00 2022] DELETE /identity/v3/projects/66456a796af547c190c9d32a2603bcdc => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:10.642018 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:11.478195 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:12.498128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:12.498128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:12.514233 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e37c1-97a1-4fc1-8aa8-420b0e73bd97 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:12.573331 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:12.573884 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:12.574164 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:12.574441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] PATH_INFO: `/v3/users/224f1253e6f24c4f9bd351657c65160e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:12.575571 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224f1253e6f24c4f9bd351657c65160e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:12.577553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:12.721201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224f1253e6f24c4f9bd351657c65160e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:12.733428 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224f1253e6f24c4f9bd351657c65160e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:12.741094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224f1253e6f24c4f9bd351657c65160e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:12.801350 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224f1253e6f24c4f9bd351657c65160e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:12.802019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224f1253e6f24c4f9bd351657c65160e'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:12.809222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1417/2829] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:44:12 2022] DELETE /identity/v3/users/224f1253e6f24c4f9bd351657c65160e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:12.813738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:12.814219 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:12.814318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:12.834120 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:12.835485 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:12.893820 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1413/2830] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:44:12.904590 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:12.906049 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:12.968403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc748c-2814-433b-9a0f-a9b7ce49abf2 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:12.972130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ac0fc-f7f7-4fb2-9983-804381427149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:12.972769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ac0fc-f7f7-4fb2-9983-804381427149 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:12.973046 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ac0fc-f7f7-4fb2-9983-804381427149 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:12.973323 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ac0fc-f7f7-4fb2-9983-804381427149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:12.982935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ac0fc-f7f7-4fb2-9983-804381427149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:12.984385 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ac0fc-f7f7-4fb2-9983-804381427149 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:12.994945 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1418/2831] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:13.422648 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:13.424154 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:13.436946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc193dc-5256-4b5a-ad8d-6bf82ac22d4a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:13.441601 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:13.442261 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:13.442543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:13.442810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] PATH_INFO: `/v3/projects/423943e4de9b4b389cc6d38a10a525ed` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:13.444864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423943e4de9b4b389cc6d38a10a525ed)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:13.446715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:13.673065 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423943e4de9b4b389cc6d38a10a525ed', 'request_id': 'req-7e1bfe4f-1e25-4c84-8362-c24523348c33'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:13.681020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423943e4de9b4b389cc6d38a10a525ed', 'request_id': 'req-7e1bfe4f-1e25-4c84-8362-c24523348c33'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:13.688046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423943e4de9b4b389cc6d38a10a525ed', 'request_id': 'req-7e1bfe4f-1e25-4c84-8362-c24523348c33'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:13.775961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] The token cache is being invalidate because project 423943e4de9b4b389cc6d38a10a525ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:44:13.776244 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:13.776976 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-7e1bfe4f-1e25-4c84-8362-c24523348c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:13.783705 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1414/2832] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:44:13 2022] DELETE /identity/v3/projects/423943e4de9b4b389cc6d38a10a525ed => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:13.784678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:13.785146 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:13.785251 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:13.806916 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:13.808231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:13.838225 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:13.887432 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1419/2833] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:44:13.897513 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:13.898111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:13.898719 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:13.919792 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:13.921069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:13.932226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1415/2834] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:44:13.934663 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:13.945898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f879abf-d8f6-49ff-b4db-d16af676d4c0 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:13.950789 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:13.951422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:13.951698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:13.951971 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:13.952652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:44:13.952962 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:13.954566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:13.968262 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1420/2835] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:44:13.973312 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:13.983408 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a336c-0735-41c1-be58-bff4b836426d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:13.988137 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:13.988956 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:13.989232 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:13.989498 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:13.990119 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:13.992207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:14.102285 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1416/2836] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:44:13 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:44:14.106742 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:14.120747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa541a-da55-412c-a9e7-aa6297fac524 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:14.182617 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:14.183158 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:14.183437 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:14.183704 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:14.184542 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:14.186626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:14.297052 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1421/2837] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:44:14 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:44:14.299738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:14.310637 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe713b4b-c52c-48f6-b277-cedb0a439ef1 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:14.315591 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:14.316194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:14.316522 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:14.316790 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:14.317429 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:14.319879 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:14.330248 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1417/2838] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:44:14.334788 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:14.345165 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6790573-a654-40e1-95d2-f5ea3dd49255 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:14.349849 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ba262-1b8c-4993-8b35-63dab0faeee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:14.350777 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ba262-1b8c-4993-8b35-63dab0faeee8 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:14.351036 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ba262-1b8c-4993-8b35-63dab0faeee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:14.351305 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ba262-1b8c-4993-8b35-63dab0faeee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:14.351942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ba262-1b8c-4993-8b35-63dab0faeee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:14.353555 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ba262-1b8c-4993-8b35-63dab0faeee8 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:14.364245 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1422/2839] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:44:14.368784 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:14.378774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c922b0-a86b-43ac-9153-039b2c3f688c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:14.384733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:14.385389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:14.385666 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:14.385934 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] PATH_INFO: `/v3/projects/f4aa358ff1064f05968fff832ea588ee/users/282cf38c59c4445ca943236cd9a08ec6/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:14.401757 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4aa358ff1064f05968fff832ea588ee, user_id=282cf38c59c4445ca943236cd9a08ec6, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:14.404437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:14.506928 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1418/2840] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:44:14 2022] PUT /identity/v3/projects/f4aa358ff1064f05968fff832ea588ee/users/282cf38c59c4445ca943236cd9a08ec6/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:14.521041 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25b11-49df-4e2e-b6bd-3971d9373364 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:14.521320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25b11-49df-4e2e-b6bd-3971d9373364 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:14.522480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25b11-49df-4e2e-b6bd-3971d9373364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:14.550396 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-83c25b11-49df-4e2e-b6bd-3971d9373364 None None] MFA Rules not processed for user `282cf38c59c4445ca943236cd9a08ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:14.551606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-83c25b11-49df-4e2e-b6bd-3971d9373364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:14.618584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1423/2841] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:44:14.629463 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:14.631957 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:14.654952 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:14.692150 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddd41b-227a-410a-9aab-8d288d588664 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:14.696877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:14.697479 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:14.697756 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:14.698026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:14.706722 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:14.708193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:14.718906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1419/2842] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:24.614554 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:24.760865 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=17 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:27.038902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505d412-d2ba-4eb5-a2ab-acbe4ad1f81b None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:27.039553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505d412-d2ba-4eb5-a2ab-acbe4ad1f81b None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:27.039553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505d412-d2ba-4eb5-a2ab-acbe4ad1f81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:27.058473 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-e505d412-d2ba-4eb5-a2ab-acbe4ad1f81b None None] MFA Rules not processed for user `282cf38c59c4445ca943236cd9a08ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:27.059677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e505d412-d2ba-4eb5-a2ab-acbe4ad1f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:27.067780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1424/2843] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:44:37.052918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:47.921396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:47.922870 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:48.020461 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c5de7-1495-4b7a-9ec8-e0424c15aead service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:48.026916 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:48.027620 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:48.027901 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:48.028172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] PATH_INFO: `/v3/users/cbffc35b26af4f13bcb582d502f978f2` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:48.029718 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbffc35b26af4f13bcb582d502f978f2)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:48.031772 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:48.273630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbffc35b26af4f13bcb582d502f978f2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:48.343841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbffc35b26af4f13bcb582d502f978f2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:48.344503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbffc35b26af4f13bcb582d502f978f2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:48.358335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbffc35b26af4f13bcb582d502f978f2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:48.365483 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbffc35b26af4f13bcb582d502f978f2'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:48.371837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1420/2844] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:44:47 2022] DELETE /identity/v3/users/cbffc35b26af4f13bcb582d502f978f2 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:48.377061 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:48.377575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:48.377575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:48.396893 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:48.400520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:48.460125 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1425/2845] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:44:48.471110 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:48.475745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:48.531472 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bb206-2c46-4535-80a3-dd8b87afe8f3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:48.534938 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:48.535536 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:48.535810 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:48.536077 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:48.546391 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:48.547821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:48.557837 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1421/2846] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:44:48.964007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:48.965701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:48.982371 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075da8b0-3a0d-4ac9-9e78-ed495f418097 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.008216 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.009931 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.010417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.011004 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] PATH_INFO: `/v3/projects/b41f7b96c8474f0f89b93473eb73d7ee` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.013076 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41f7b96c8474f0f89b93473eb73d7ee)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.017275 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:49.177386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41f7b96c8474f0f89b93473eb73d7ee', 'request_id': 'req-d26e8cc0-0d02-4540-a1eb-20a0140518df'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:49.195069 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:49.205624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c424-e720-4f9a-854c-8cf9612bf2a3 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.243100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:49.295952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41f7b96c8474f0f89b93473eb73d7ee', 'request_id': 'req-d26e8cc0-0d02-4540-a1eb-20a0140518df'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:49.304040 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41f7b96c8474f0f89b93473eb73d7ee', 'request_id': 'req-d26e8cc0-0d02-4540-a1eb-20a0140518df'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:49.310279 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.311301 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.312172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.313304 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.317072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:49.326480 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] The token cache is being invalidate because project b41f7b96c8474f0f89b93473eb73d7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:44:49.327033 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:49.327911 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-d26e8cc0-0d02-4540-a1eb-20a0140518df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:49.332422 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1426/2847] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:44:48 2022] DELETE /identity/v3/projects/b41f7b96c8474f0f89b93473eb73d7ee => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:49.335440 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.335628 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.335874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.358190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:49.359489 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:49.455678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1427/2848] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:44:49.458874 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:49.462959 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:49.475741 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.477561 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133dfc0-0ab3-4eed-9592-39381fd5f5ec service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:49.489986 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff8ccc-8fff-43b8-8ef1-23e96a7b6502 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.496159 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.496815 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.497082 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.497344 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.499119 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:49.510349 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1422/2849] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:49.511770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.512067 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.512363 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.516819 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.518270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:49.533563 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1428/2850] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:49.535049 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:49.536443 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:49.544700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1423/2851] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:44:49.549282 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:49.564539 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710940-ed86-4c9b-9039-8cdb4c21af5e service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.575679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.580123 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.580468 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.580751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.581602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:44:49.581926 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.583606 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:49.592806 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:49.599285 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1429/2852] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:44:49.604153 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:49.615979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c94af-d7a6-4d2f-95e9-92ebc9fdecd3 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.621480 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.630170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.630170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.630170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.630170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.630170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:49.757933 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1424/2853] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:44:49 2022] POST /identity/v3/projects => generated 391 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:44:49.763380 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:49.775872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f1efb-8503-4aa7-ba65-b6a7e35f7cb7 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.860090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.860732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.861105 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.861386 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.863937 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.867117 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:49.966727 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1430/2854] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:44:49 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:44:49.973380 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:49.984354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ff707-a4cd-4f85-91c5-7ceee091f83d admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:49.989180 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:49.989774 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:49.990181 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:49.990490 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:49.991092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:49.992782 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:50.003566 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1425/2855] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:44:50.007948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:50.019193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564447f-b913-424f-80c8-5154888221e6 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:50.026677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4fa1-1b1b-4adf-ad29-664152f28451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:50.026677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4fa1-1b1b-4adf-ad29-664152f28451 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:50.026677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4fa1-1b1b-4adf-ad29-664152f28451 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:50.027706 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4fa1-1b1b-4adf-ad29-664152f28451 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:50.029130 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb4fa1-1b1b-4adf-ad29-664152f28451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:50.030007 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb4fa1-1b1b-4adf-ad29-664152f28451 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:50.043199 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1431/2856] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:44:50.048493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:50.059025 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd0d6b-91e8-4746-ba6b-9dd6d41085f3 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:50.063890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:50.064715 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] REQUEST_METHOD: `PUT` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:50.065164 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:50.065424 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] PATH_INFO: `/v3/projects/0f4c23369fc34f959d1963cb851148f8/users/9f0b19c1c15645348c9686c6491ecd46/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:50.089544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4c23369fc34f959d1963cb851148f8, user_id=9f0b19c1c15645348c9686c6491ecd46, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:50.092177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:50.190626 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1426/2857] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:44:50 2022] PUT /identity/v3/projects/0f4c23369fc34f959d1963cb851148f8/users/9f0b19c1c15645348c9686c6491ecd46/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:50.198412 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:50.198660 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:50.198860 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:50.231190 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] MFA Rules not processed for user `9f0b19c1c15645348c9686c6491ecd46`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:44:50.232780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:50.316534 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1432/2858] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:44:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:44:50.329949 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:50.331697 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:50.377909 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:50.406684 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65411cd-c1d9-4374-b558-c6709a3bef5c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:50.410433 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:50.411043 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:50.411316 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:50.411582 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:50.424236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:50.425823 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:50.438188 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1427/2859] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:53.142137 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:53.217710 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f049e-05eb-4352-9439-bd5ba76046ff None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:53.221340 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:53.221952 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:53.222482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:53.222752 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:53.224553 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:53.248024 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:53.293658 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:53.295299 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:53.306298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1433/2860] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:00.541100 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=12 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:01.282452 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:01.284115 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:01.300817 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf96deb-eb18-4ba0-bce5-e31572d0b97c service cinder] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:01.306847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:01.307605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:01.307884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:01.308158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] PATH_INFO: `/v3/users/022c4688d9ce4df9a99ae5fa84e1d871` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:01.309386 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=022c4688d9ce4df9a99ae5fa84e1d871)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:01.311319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:01.494268 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022c4688d9ce4df9a99ae5fa84e1d871'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:01.563484 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022c4688d9ce4df9a99ae5fa84e1d871'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:01.564359 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022c4688d9ce4df9a99ae5fa84e1d871'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:01.578971 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022c4688d9ce4df9a99ae5fa84e1d871'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:01.589156 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022c4688d9ce4df9a99ae5fa84e1d871'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:01.596891 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1428/2861] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:45:01 2022] DELETE /identity/v3/users/022c4688d9ce4df9a99ae5fa84e1d871 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:45:01.860346 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:01.875030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b7c74-3ebe-40d4-b2ab-38184d5e9e0d service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:01.940261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:01.940922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:01.941201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:01.941500 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] PATH_INFO: `/v3/projects/ebe5283d88b44f8b824c7bbca56b3473` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:01.942716 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe5283d88b44f8b824c7bbca56b3473)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:01.944688 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:02.088725 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe5283d88b44f8b824c7bbca56b3473', 'request_id': 'req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.190388 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe5283d88b44f8b824c7bbca56b3473', 'request_id': 'req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.199022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe5283d88b44f8b824c7bbca56b3473', 'request_id': 'req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.210950 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] The token cache is being invalidate because project ebe5283d88b44f8b824c7bbca56b3473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:45:02.211289 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.212231 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.213755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1434/2862] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:45:01 2022] DELETE /identity/v3/projects/ebe5283d88b44f8b824c7bbca56b3473 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:02.217877 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:02.219334 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:02.235302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4848-0acd-4d2b-9100-91428e9575d7 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:02.299583 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:02.302605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:02.302605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:02.302605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] PATH_INFO: `/v3/users/bc49646f0846434f9fa3931b5db0f517` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:02.302605 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc49646f0846434f9fa3931b5db0f517)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:02.303873 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:02.500905 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc49646f0846434f9fa3931b5db0f517'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.589275 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc49646f0846434f9fa3931b5db0f517'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.589951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc49646f0846434f9fa3931b5db0f517'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.602747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc49646f0846434f9fa3931b5db0f517'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.604743 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:02.611168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-2b059198-8a30-47fc-b57b-1bd787f704a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc49646f0846434f9fa3931b5db0f517'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:02.617465 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1429/2863] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:45:02 2022] DELETE /identity/v3/users/bc49646f0846434f9fa3931b5db0f517 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:02.859636 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:02.861116 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:02.877780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b4d4e-db74-4850-aed7-1a0d55bf58cd admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:02.925180 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:02.939641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:02.940501 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:02.940656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:02.940934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] PATH_INFO: `/v3/projects/79152a9312394a609050dbeb63c36310` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:02.942080 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] RBAC: Authorizing `identity:delete_project(project_id=79152a9312394a609050dbeb63c36310)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:02.944019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:03.070208 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79152a9312394a609050dbeb63c36310', 'request_id': 'req-789780dc-ae57-4962-8fc4-a693a146550a'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:03.161063 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79152a9312394a609050dbeb63c36310', 'request_id': 'req-789780dc-ae57-4962-8fc4-a693a146550a'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:03.167878 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79152a9312394a609050dbeb63c36310', 'request_id': 'req-789780dc-ae57-4962-8fc4-a693a146550a'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:03.179005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] The token cache is being invalidate because project 79152a9312394a609050dbeb63c36310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:45:03.182400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:03.182400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-789780dc-ae57-4962-8fc4-a693a146550a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:03.182400 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1435/2864] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:45:02 2022] DELETE /identity/v3/projects/79152a9312394a609050dbeb63c36310 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:12.474229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:12.474775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:12.474775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:12.496470 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] MFA Rules not processed for user `4c936369a3ff4138833cfc81a1104f20`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:12.497733 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:12.552751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1430/2865] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:13.176072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:13.287626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979a01-5793-4a7c-a5cd-35258889346f None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:13.287936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979a01-5793-4a7c-a5cd-35258889346f None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:13.288230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b979a01-5793-4a7c-a5cd-35258889346f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:13.353013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-3b979a01-5793-4a7c-a5cd-35258889346f None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:13.353302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-3b979a01-5793-4a7c-a5cd-35258889346f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:13.435733 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1436/2866] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:45:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:13.594131 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:13.598346 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:13.715020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46141b5c-7c2d-4932-b376-e05b682d814a None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:13.720198 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:13.720841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:13.721118 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:13.721395 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:13.730148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:13.731745 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:13.743106 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1431/2867] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:45:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:45:14.662964 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:14.663511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:14.663511 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:14.684913 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:14.686134 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:14.693972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1437/2868] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 00:45:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:14.706902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:14.719898 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b9a93-0f3d-4677-959e-c6d81ef9093e service swift] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:14.726072 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4edc20-7f39-49f5-a41d-5e085d4a233c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:14.726658 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4edc20-7f39-49f5-a41d-5e085d4a233c service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:14.727477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4edc20-7f39-49f5-a41d-5e085d4a233c service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:14.727746 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4edc20-7f39-49f5-a41d-5e085d4a233c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:14.737568 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4edc20-7f39-49f5-a41d-5e085d4a233c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:14.739350 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4edc20-7f39-49f5-a41d-5e085d4a233c service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:14.748597 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1432/2869] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:45:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:45:14.952717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:14.954963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:15.038923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff8cc-4ab2-42e8-9c92-a9f0b2905707 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:15.043743 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:15.044413 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:15.044699 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:15.044998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] PATH_INFO: `/v3/users/9f0b19c1c15645348c9686c6491ecd46` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:15.046861 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0b19c1c15645348c9686c6491ecd46)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:15.048829 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:15.245583 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0b19c1c15645348c9686c6491ecd46'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:15.257701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0b19c1c15645348c9686c6491ecd46'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:15.263258 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:15.266182 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0b19c1c15645348c9686c6491ecd46'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:15.358614 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0b19c1c15645348c9686c6491ecd46'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:15.359316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0b19c1c15645348c9686c6491ecd46'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:15.366575 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1438/2870] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:45:14 2022] DELETE /identity/v3/users/9f0b19c1c15645348c9686c6491ecd46 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:15.371728 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4e4f2-e122-4e4c-af4f-7d323ef86247 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:15.372036 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4e4f2-e122-4e4c-af4f-7d323ef86247 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:15.372281 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4e4f2-e122-4e4c-af4f-7d323ef86247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:15.393108 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8ba4e4f2-e122-4e4c-af4f-7d323ef86247 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:15.394300 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8ba4e4f2-e122-4e4c-af4f-7d323ef86247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:15.449254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=100 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:15.461439 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1433/2871] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:15.471977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:15.473677 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:15.505698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:15.506260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:15.506535 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:15.529240 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:15.530501 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:15.558458 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2fbf8-bcb5-476a-8260-4ef0f84bc5b3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:15.562498 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:15.563122 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:15.563469 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:15.563674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:15.575109 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:15.576603 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:15.593470 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1439/2872] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:15.603246 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1434/2873] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:45:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:15.621639 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:15.623441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:15.721270 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f84c35-4230-423f-900e-39f02d3fcb26 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:15.727701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b1bd5-97f1-4289-8ea1-c616d39dbcf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:15.728630 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1bd5-97f1-4289-8ea1-c616d39dbcf9 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:15.730709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1bd5-97f1-4289-8ea1-c616d39dbcf9 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:15.730709 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1bd5-97f1-4289-8ea1-c616d39dbcf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:15.738653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b1bd5-97f1-4289-8ea1-c616d39dbcf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:15.740250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b1bd5-97f1-4289-8ea1-c616d39dbcf9 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:15.757091 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1440/2874] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:45:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:45:16.091330 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.092850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.122031 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6100d1-8aea-4b79-90a3-c9a4b8a8de93 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.126999 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.127771 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.128236 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.128477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] PATH_INFO: `/v3/projects/0f4c23369fc34f959d1963cb851148f8` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.129703 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4c23369fc34f959d1963cb851148f8)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.131677 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.307192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4c23369fc34f959d1963cb851148f8', 'request_id': 'req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:16.313587 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4c23369fc34f959d1963cb851148f8', 'request_id': 'req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:16.321678 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4c23369fc34f959d1963cb851148f8', 'request_id': 'req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:16.387481 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] The token cache is being invalidate because project 0f4c23369fc34f959d1963cb851148f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:45:16.391897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:16.391897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-eaa772ad-6efa-4dd5-b7c8-13cdb0e89546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:16.391897 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1435/2875] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:45:16 2022] DELETE /identity/v3/projects/0f4c23369fc34f959d1963cb851148f8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:16.395516 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a6d0-b778-4113-a85f-044f15acc435 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.395802 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a6d0-b778-4113-a85f-044f15acc435 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.396073 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a6d0-b778-4113-a85f-044f15acc435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.409092 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.409968 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.410260 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.421014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-aa74a6d0-b778-4113-a85f-044f15acc435 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:16.422515 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-aa74a6d0-b778-4113-a85f-044f15acc435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.434994 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:16.436227 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.494255 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1441/2876] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:45:16.502322 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.502609 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.502881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.504538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1436/2877] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 00:45:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:45:16.520489 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.521915 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.525565 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:16.526821 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.536240 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1442/2878] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:45:16.541717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.559079 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a7808-8a72-4936-8b0c-483de950fd91 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.561492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:16.568525 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.569584 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.570402 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.571205 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] PATH_INFO: `/v3/domains` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.572230 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:45:16.572631 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.574820 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.584260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:16.593406 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1443/2879] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:45:16.608104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.621635 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95ff3-66d1-4c44-8a9e-1c01a57a5961 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.622207 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c664f2-4ce4-408c-bb61-e7b114bfa1f6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.628859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.629087 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.629546 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.629745 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.629978 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.630152 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.630269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.630547 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.631358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.635483 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.645615 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.649964 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.662389 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1437/2880] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:45:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 00:45:16.754410 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1444/2881] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:45:16 2022] POST /identity/v3/projects => generated 397 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:45:16.762701 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.775898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.778675 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.779602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020d2c2-bd88-4cc4-b9c1-72e4b1d33029 service swift] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.855550 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a101faa-bc89-4387-bc9b-aff8f354b974 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.859005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.859550 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.859680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.859858 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.860130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.860253 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.860566 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.860789 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.860902 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.862650 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:16.862775 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.874324 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.877424 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.885003 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1445/2882] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:16.938347 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1438/2883] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:45:16 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:45:16.942517 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.954920 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bad8bc-0026-4d87-9826-268eafc19017 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:16.960013 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:16.960648 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:16.960978 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:16.961268 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:16.962193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:16.963780 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:16.974261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1446/2884] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:45:16.985920 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:16.996936 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e91a8-0c84-4063-a935-d1f67933ea56 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:17.001864 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ac49a-4030-440b-b28e-fcb6e8ef9c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:17.002453 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ac49a-4030-440b-b28e-fcb6e8ef9c0c admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:17.002730 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ac49a-4030-440b-b28e-fcb6e8ef9c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:17.003127 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ac49a-4030-440b-b28e-fcb6e8ef9c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:17.004021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ac49a-4030-440b-b28e-fcb6e8ef9c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:17.005808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ac49a-4030-440b-b28e-fcb6e8ef9c0c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:17.024531 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1439/2885] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:45:17.032746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:17.044392 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246afc-0ab0-4c22-96bc-2405a5beb8e4 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:17.058022 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:17.058320 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:17.058610 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:17.058889 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] PATH_INFO: `/v3/projects/15661a180ec04cee80d32794e4079655/users/f25e49e1029b49e3908c3753556c36b7/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:17.081450 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15661a180ec04cee80d32794e4079655, user_id=f25e49e1029b49e3908c3753556c36b7, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:17.083770 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:17.159865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1447/2886] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:45:17 2022] PUT /identity/v3/projects/15661a180ec04cee80d32794e4079655/users/f25e49e1029b49e3908c3753556c36b7/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:17.172142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:17.172649 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:17.172751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:17.208731 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] MFA Rules not processed for user `f25e49e1029b49e3908c3753556c36b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:17.210241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:17.303500 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1440/2887] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:17.313674 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:17.315533 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:17.381291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846471-d94d-4845-83f5-8c578c015109 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:17.386854 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2466c6d-7048-4144-8c24-e17e8bca4047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a765c1f74578435d8f29fbf959e5f087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:17.387485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2466c6d-7048-4144-8c24-e17e8bca4047 service cinder] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:17.387835 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2466c6d-7048-4144-8c24-e17e8bca4047 service cinder] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:17.388161 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2466c6d-7048-4144-8c24-e17e8bca4047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:17.397680 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2466c6d-7048-4144-8c24-e17e8bca4047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:17.399518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2466c6d-7048-4144-8c24-e17e8bca4047 service cinder] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:17.409670 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1448/2888] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:45:19.578007 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:19.579492 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:19.607979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:19.646569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dd0f2-5b3b-4962-9c2e-c030b0a5b5cd None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:19.652005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:19.655060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:19.655060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:19.655060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] PATH_INFO: `/v3/users/282cf38c59c4445ca943236cd9a08ec6` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:19.655060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=282cf38c59c4445ca943236cd9a08ec6)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:19.656193 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:19.913239 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282cf38c59c4445ca943236cd9a08ec6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.014569 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282cf38c59c4445ca943236cd9a08ec6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.015190 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282cf38c59c4445ca943236cd9a08ec6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.029111 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282cf38c59c4445ca943236cd9a08ec6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.037360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282cf38c59c4445ca943236cd9a08ec6'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.044018 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1441/2889] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:45:19 2022] DELETE /identity/v3/users/282cf38c59c4445ca943236cd9a08ec6 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:20.048726 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:20.049179 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:20.049278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:20.086188 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:20.087546 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:20.132298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:20.152864 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1449/2890] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:20.163390 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:20.165469 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:20.223989 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0f4a5-9dbb-479a-a068-91de72625d93 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:20.228271 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d6c4d-99d3-4f4d-b751-fa831a2899c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:20.228893 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d6c4d-99d3-4f4d-b751-fa831a2899c1 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:20.229172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d6c4d-99d3-4f4d-b751-fa831a2899c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:20.229437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d6c4d-99d3-4f4d-b751-fa831a2899c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:20.239058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d6c4d-99d3-4f4d-b751-fa831a2899c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:20.240518 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d6c4d-99d3-4f4d-b751-fa831a2899c1 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:20.251477 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1442/2891] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:20.616751 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:20.618825 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:20.634442 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b768f-ea39-419b-9049-f5e3128c7bde None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:20.639764 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:20.640862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:20.640862 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:20.649178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] PATH_INFO: `/v3/projects/f4aa358ff1064f05968fff832ea588ee` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:20.649178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4aa358ff1064f05968fff832ea588ee)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:20.649178 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:20.749430 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4aa358ff1064f05968fff832ea588ee', 'request_id': 'req-e20194ba-d5a3-4c67-b469-051095d33345'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.834921 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4aa358ff1064f05968fff832ea588ee', 'request_id': 'req-e20194ba-d5a3-4c67-b469-051095d33345'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.841948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4aa358ff1064f05968fff832ea588ee', 'request_id': 'req-e20194ba-d5a3-4c67-b469-051095d33345'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.857747 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] The token cache is being invalidate because project f4aa358ff1064f05968fff832ea588ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:45:20.858031 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.858678 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-e20194ba-d5a3-4c67-b469-051095d33345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:20.860145 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1450/2892] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:45:20 2022] DELETE /identity/v3/projects/f4aa358ff1064f05968fff832ea588ee => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:20.867537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:20.868158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:20.868158 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:20.888690 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:20.889973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:20.933314 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:20.951020 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1443/2893] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:20.959615 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:20.959896 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:20.960189 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:20.981202 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:20.982425 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:20.990846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1451/2894] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:20.995521 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.005622 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe13aa1-32e9-4f02-8ef3-41d0264052fb None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.011097 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.011698 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.011979 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.012252 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] PATH_INFO: `/v3/domains` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.012919 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:45:21.013226 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.014875 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.027808 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1444/2895] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 00:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:45:21.032698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.042403 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20834826-f12c-404b-9ad5-1971d3d91b9c None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.047362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.047977 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.048261 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.048585 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] PATH_INFO: `/v3/projects` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.049197 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.051114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.154700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1452/2896] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:45:21 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:45:21.159090 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.169770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd574b-be67-4dd1-8c16-6411f028be00 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.226641 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.227241 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.227514 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.227780 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] PATH_INFO: `/v3/users` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.228437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.230360 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.352021 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1445/2897] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:45:21 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:45:21.356155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.367552 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e17d69-0691-4b5b-b793-fc1bde3b679b admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.372278 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.372983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.373291 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.373577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.374313 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.375903 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.388030 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1453/2898] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:45:21.392048 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.402969 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73cc1-f6fa-494f-868e-ef0b4511fe2c admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef464d-eb0f-4d78-b45c-b8aee081a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.408602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef464d-eb0f-4d78-b45c-b8aee081a867 admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.408876 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef464d-eb0f-4d78-b45c-b8aee081a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.409142 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef464d-eb0f-4d78-b45c-b8aee081a867 admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.409785 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef464d-eb0f-4d78-b45c-b8aee081a867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.411388 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef464d-eb0f-4d78-b45c-b8aee081a867 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.423419 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1446/2899] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:45:21.429193 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.439830 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189bf0-03e8-4392-aad4-c3284a272a6c admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.444429 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.445898 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.446163 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.446441 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] PATH_INFO: `/v3/projects/3737c99210b64eac8d7d75f53c1e2cac/users/23fc9f06d2bb4a0fbc0fc01dda38b522/roles/c326d7ede2124b9e9b3714569c80f57e` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.461183 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3737c99210b64eac8d7d75f53c1e2cac, user_id=23fc9f06d2bb4a0fbc0fc01dda38b522, role_id=c326d7ede2124b9e9b3714569c80f57e)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.464086 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.576099 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1454/2900] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:45:21 2022] PUT /identity/v3/projects/3737c99210b64eac8d7d75f53c1e2cac/users/23fc9f06d2bb4a0fbc0fc01dda38b522/roles/c326d7ede2124b9e9b3714569c80f57e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:21.581339 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.581721 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.582063 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.603770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:21.605058 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:21.669464 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1447/2901] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:21.679276 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:21.681870 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:21.739446 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89790e-5ac3-4c7b-8692-dc75357b8af0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:21.743857 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb782013-bdc9-45c4-83ce-804fc7173758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:21.744577 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb782013-bdc9-45c4-83ce-804fc7173758 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:21.744859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb782013-bdc9-45c4-83ce-804fc7173758 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:21.745131 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb782013-bdc9-45c4-83ce-804fc7173758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:21.754472 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb782013-bdc9-45c4-83ce-804fc7173758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:21.755890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb782013-bdc9-45c4-83ce-804fc7173758 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:21.765934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1455/2902] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:26.983760 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:26.985042 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:26.997881 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744ff3-cf25-4208-9db8-ff8a4150910d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:27.002986 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:27.003628 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:27.003902 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:27.004172 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] PATH_INFO: `/v3/users/f25e49e1029b49e3908c3753556c36b7` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:27.005413 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25e49e1029b49e3908c3753556c36b7)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:27.007387 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:27.248159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25e49e1029b49e3908c3753556c36b7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:27.250124 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:27.444726 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25e49e1029b49e3908c3753556c36b7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:27.446511 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25e49e1029b49e3908c3753556c36b7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:27.465430 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25e49e1029b49e3908c3753556c36b7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:27.474896 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-3636746b-32ef-49e7-beb5-3059a4a81850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25e49e1029b49e3908c3753556c36b7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:27.486282 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1448/2903] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:45:26 2022] DELETE /identity/v3/users/f25e49e1029b49e3908c3753556c36b7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:27.490599 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:27.492564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:27.492564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:27.516872 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] MFA Rules not processed for user `3b2336264fcc4528b59320d71b7da06f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:27.528303 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:27.541244 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:27.553584 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:27.553951 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:27.554926 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:27.592831 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] MFA Rules not processed for user `23fc9f06d2bb4a0fbc0fc01dda38b522`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:27.594103 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:27.628528 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1456/2904] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:27.640602 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:27.642358 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:27.677884 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1449/2905] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:27.683543 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:27.685005 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:27.700039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d0af4-12aa-404d-8209-b7dec5fd09c6 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:27.705080 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:27.705700 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:27.705981 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:27.706256 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:27.706900 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:27.708972 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:27.720597 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c081485-cab4-4d6c-9149-be136b3aa1df None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:27.731155 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:27.731792 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:27.732070 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:27.732405 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:27.750914 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:27.751983 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:27.764169 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1457/2906] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:45:27.832405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1450/2907] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 00:45:27 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:45:27.837081 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:27.851384 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2ccd3-8abf-4d96-8abe-1cfe249c3737 service neutron] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:27.928362 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:27.929128 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:27.929532 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:27.929918 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] PATH_INFO: `/v3/users` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:27.930656 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:27.932701 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:28.068302 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1458/2908] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 00:45:27 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:45:28.069212 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:28.082045 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e1e97-e903-4dda-9d27-4d51444b1fe4 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:28.087769 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:28.090805 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:28.091076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:28.091564 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] PATH_INFO: `/v3/roles` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:28.092493 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:28.094234 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:28.109394 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1451/2909] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:45:28.110332 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:28.122968 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bbc77-b8af-487e-a315-fe6a3fe163c3 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:28.128213 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:28.129005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:28.129292 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:28.129564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] PATH_INFO: `/v3/projects/2a64b8377ee74c21b0198dbef64c23c2/users/f9cd92b0ce17457cae931eca0b6c4c95/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:28.147541 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a64b8377ee74c21b0198dbef64c23c2, user_id=f9cd92b0ce17457cae931eca0b6c4c95, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:28.151104 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:28.252450 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:28.254201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1459/2910] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:45:28 2022] PUT /identity/v3/projects/2a64b8377ee74c21b0198dbef64c23c2/users/f9cd92b0ce17457cae931eca0b6c4c95/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:28.254354 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:28.259703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:28.282830 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6a9b-0739-404f-a898-21fd88d6526b admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:28.335551 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23d507-4b87-4625-8a2d-1cad2ed1355d admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:28.340769 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:28.341652 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:28.342094 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:28.342357 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] PATH_INFO: `/v3/roles` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:28.343201 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:28.346277 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:28.351046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:28.353265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:28.353616 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:28.353961 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] PATH_INFO: `/v3/projects/15661a180ec04cee80d32794e4079655` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:28.356123 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] RBAC: Authorizing `identity:delete_project(project_id=15661a180ec04cee80d32794e4079655)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:28.357999 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1460/2911] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 00:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:45:28.358647 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:28.372520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:28.399107 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51970ec5-066b-4b18-9f6c-ef7f334b8abd admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:28.405935 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d02e24-58ec-49f8-a2dc-48843cc4b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:28.407085 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02e24-58ec-49f8-a2dc-48843cc4b860 admin admin] REQUEST_METHOD: `PUT` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:28.407589 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02e24-58ec-49f8-a2dc-48843cc4b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:28.407963 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02e24-58ec-49f8-a2dc-48843cc4b860 admin admin] PATH_INFO: `/v3/projects/2a64b8377ee74c21b0198dbef64c23c2/users/f9cd92b0ce17457cae931eca0b6c4c95/roles/ac2134ae61074ce6b751deadce042c42` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:28.410260 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d02e24-58ec-49f8-a2dc-48843cc4b860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a64b8377ee74c21b0198dbef64c23c2, user_id=f9cd92b0ce17457cae931eca0b6c4c95, role_id=ac2134ae61074ce6b751deadce042c42)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:28.413356 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d02e24-58ec-49f8-a2dc-48843cc4b860 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:28.430700 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1461/2912] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 00:45:28 2022] PUT /identity/v3/projects/2a64b8377ee74c21b0198dbef64c23c2/users/f9cd92b0ce17457cae931eca0b6c4c95/roles/ac2134ae61074ce6b751deadce042c42 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:45:28.510393 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15661a180ec04cee80d32794e4079655', 'request_id': 'req-4c95e6bf-1bfc-453c-ab50-8e5badd205af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:28.515973 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15661a180ec04cee80d32794e4079655', 'request_id': 'req-4c95e6bf-1bfc-453c-ab50-8e5badd205af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:28.522293 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15661a180ec04cee80d32794e4079655', 'request_id': 'req-4c95e6bf-1bfc-453c-ab50-8e5badd205af'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:28.681026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] The token cache is being invalidate because project 15661a180ec04cee80d32794e4079655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:45:28.681398 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:28.682094 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:45:28.683675 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1452/2913] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:45:28 2022] DELETE /identity/v3/projects/15661a180ec04cee80d32794e4079655 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:45:29.923503 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:INSERT=2,keystone:DELETE=4,keystone:SELECT=94 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:30.172925 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:34.850586 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc6743-4fb6-4e83-ba51-71e3f73d74e1 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:34.851408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc6743-4fb6-4e83-ba51-71e3f73d74e1 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:34.851408 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc6743-4fb6-4e83-ba51-71e3f73d74e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:34.882996 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-2dfc6743-4fb6-4e83-ba51-71e3f73d74e1 None None] MFA Rules not processed for user `f9cd92b0ce17457cae931eca0b6c4c95`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:34.884359 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-2dfc6743-4fb6-4e83-ba51-71e3f73d74e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:34.963252 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1462/2914] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:45:34.975214 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:34.976841 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:35.040000 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95e6bf-1bfc-453c-ab50-8e5badd205af admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:35.044177 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad5664c09ed44105a1083cc90b3b739a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:35.044821 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:35.045101 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:35.045525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:35.047265 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:35.110890 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:35.113355 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:35.124403 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1453/2915] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:35.352813 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f310e22-f6b8-4897-a3cf-dd688018f2c9 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:35.353235 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f310e22-f6b8-4897-a3cf-dd688018f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:35.353507 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f310e22-f6b8-4897-a3cf-dd688018f2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:35.375045 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-5f310e22-f6b8-4897-a3cf-dd688018f2c9 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:35.376263 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-5f310e22-f6b8-4897-a3cf-dd688018f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:35.438250 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1463/2916] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:45:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:35.452908 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:35.454832 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:35.543946 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b96240-cf5a-4b0f-8625-c55c3f688203 service glance] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:35.550671 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:35.551402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:35.551738 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:35.552130 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:35.560850 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:35.563712 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:35.565508 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:35.585254 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1454/2917] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:45:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:45:45.431019 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:45.968930 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=2 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:46.110746 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] REQUEST_METHOD: `POST` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:46.111025 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:46.111298 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:46.130783 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.auth.core [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] MFA Rules not processed for user `23fc9f06d2bb4a0fbc0fc01dda38b522`. Rule list: `[]` (Enabled: `True`). {{(pid=88219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:46.131972 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:46.138967 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1464/2918] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 00:45:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:46.400537 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:46.401960 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:46.460046 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a08d94-f357-4579-99dd-2f39d49e17f2 service swift] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:46.464496 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:46.465105 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:46.465384 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:46.465655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:46.467437 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:46.477335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:46.478806 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb2c56-9126-410a-b371-81d40e01bb13 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:46.487371 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1455/2919] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:45:46.891653 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:46.893160 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:46.972666 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dea966-6098-4af7-b16e-0c85687b6207 None None] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:46.978545 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:46.979207 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:46.979485 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:46.979753 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:46.981561 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:46.998181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:47.000409 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:47.012797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1465/2920] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:50.132661 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:50.133669 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:50.133770 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:50.158814 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:45:50.160076 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:50.169039 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1456/2921] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 00:45:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:45:50.181936 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:50.194353 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e9205-7b92-4f17-81f7-1c9faf39a95d service placement] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:50.200881 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:50.201494 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:50.201774 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:50.202048 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:50.210998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:50.213703 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:50.222308 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1466/2922] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:45:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:45:59.157786 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:59.160169 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:59.239004 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63748dd7-cd83-46a6-bfd8-ae677459889d None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:59.244456 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:59.245060 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:59.245335 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:59.245602 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:59.247336 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:59.260548 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:59.262170 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:59.273294 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1457/2923] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:46:00.218758 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=41 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:46:05.234816 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:46:05.236558 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:46:05.255801 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9ef2d-eb51-4ffe-a2da-330556cc3e59 service swift] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:05.262271 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:05.263144 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:05.263492 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:05.263839 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:05.265797 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:46:05.279890 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:05.281461 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:05.292966 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1467/2924] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:46:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:46:09.268382 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=63 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:46:15.287865 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=6 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:46:20.811319 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:46:20.820922 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41846096-dd5a-41d1-93b9-1e169953af52 service nova] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:20.825029 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f5987c024d44acbc7bf1502f11848f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:20.825906 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:20.826194 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:20.826544 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:20.835750 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:20.837229 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:20.847847 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1458/2925] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:46:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:46:25.780222 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:46:25.795859 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a649a-ef0c-4523-bfc3-c81ba1b83455 service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:25.810601 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1427404350643b2892469b0da65b788', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:25.811236 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:25.811518 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:25.811793 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:25.813690 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:46:25.836958 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:25.838942 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:25.852133 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1468/2926] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:46:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:46:30.843866 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=6 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:46:31.840660 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:46:31.842044 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:46:31.911630 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12f023-683c-4d61-a560-37ecf418c4e0 service neutron] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:31.916576 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:31.918802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:31.918802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:31.918802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] PATH_INFO: `/v3/users/4c936369a3ff4138833cfc81a1104f20` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:31.918802 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c936369a3ff4138833cfc81a1104f20)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:31.920614 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:32.110524 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c936369a3ff4138833cfc81a1104f20'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.168224 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c936369a3ff4138833cfc81a1104f20'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.168885 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c936369a3ff4138833cfc81a1104f20'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.181525 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c936369a3ff4138833cfc81a1104f20'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.188655 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-93aaf74e-c1ce-446a-b696-b8b92e1a8684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c936369a3ff4138833cfc81a1104f20'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.194595 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1459/2927] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:46:31 2022] DELETE /identity/v3/users/4c936369a3ff4138833cfc81a1104f20 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:46:32.439732 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:46:32.453417 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96211ed-1842-426d-9d3c-81585526db4d service nova] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:32.519991 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:32.520661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:32.520934 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:32.521203 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] PATH_INFO: `/v3/projects/644ca4166ff846fb944c2b39f0263abe` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:32.522564 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644ca4166ff846fb944c2b39f0263abe)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:32.524482 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:32.634176 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644ca4166ff846fb944c2b39f0263abe', 'request_id': 'req-44c09c89-85ab-49bc-84b2-721414191d81'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.696998 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644ca4166ff846fb944c2b39f0263abe', 'request_id': 'req-44c09c89-85ab-49bc-84b2-721414191d81'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.703990 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644ca4166ff846fb944c2b39f0263abe', 'request_id': 'req-44c09c89-85ab-49bc-84b2-721414191d81'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.715641 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] The token cache is being invalidate because project 644ca4166ff846fb944c2b39f0263abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:46:32.715923 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.716582 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:46:32.718901 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1469/2928] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:46:32 2022] DELETE /identity/v3/projects/644ca4166ff846fb944c2b39f0263abe => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:46:42.191160 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:46:42.712735 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:47:51.280402 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] REQUEST_METHOD: `POST` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:47:51.280940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:47:51.280940 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:47:51.301444 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.auth.core [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] MFA Rules not processed for user `589534f84a6143a2a194ccbd1c84373f`. Rule list: `[]` (Enabled: `True`). {{(pid=88218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:47:51.302659 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:51.362853 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1460/2929] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 00:47:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:47:51.411593 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:47:51.413072 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:51.489981 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c09c89-85ab-49bc-84b2-721414191d81 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:47:51.495676 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1468fb64bc6d4b1899be323c0116c8e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:47:51.496335 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] REQUEST_METHOD: `GET` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:47:51.496627 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:47:51.496900 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:47:51.507223 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:47:51.508853 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:47:51.519600 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1470/2930] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 00:47:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 00:47:56.247827 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:47:56.249343 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:56.322328 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7c5fd-76f9-4657-a850-589e2f442354 None None] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:47:56.328405 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:47:56.329159 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:47:56.330148 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:47:56.330674 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:47:56.333098 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:56.350064 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:47:56.351577 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:47:56.365451 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1461/2931] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:47:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:48:00.198698 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:00.200665 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:00.280481 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92f166-a42d-431e-85b7-db0144a9ac27 service swift] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:00.286014 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:00.286684 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:00.286965 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:00.291397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] PATH_INFO: `/v3/users/23fc9f06d2bb4a0fbc0fc01dda38b522` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:00.291397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23fc9f06d2bb4a0fbc0fc01dda38b522)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:00.291397 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:00.529536 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23fc9f06d2bb4a0fbc0fc01dda38b522'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:00.546736 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23fc9f06d2bb4a0fbc0fc01dda38b522'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:00.554379 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23fc9f06d2bb4a0fbc0fc01dda38b522'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:00.746661 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23fc9f06d2bb4a0fbc0fc01dda38b522'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:00.747311 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23fc9f06d2bb4a0fbc0fc01dda38b522'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:00.754922 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1471/2932] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:48:00 2022] DELETE /identity/v3/users/23fc9f06d2bb4a0fbc0fc01dda38b522 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:48:01.070467 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:01.094104 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8f17f-d5b7-47c4-a777-59cacee58f54 service placement] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:01.164571 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:01.165197 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:01.165471 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:01.165737 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] PATH_INFO: `/v3/projects/3737c99210b64eac8d7d75f53c1e2cac` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:01.167192 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3737c99210b64eac8d7d75f53c1e2cac)` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:01.169396 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:01.410288 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:48:01.420635 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3737c99210b64eac8d7d75f53c1e2cac', 'request_id': 'req-cea0df24-45bc-4881-898c-d21769b756f7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.426625 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3737c99210b64eac8d7d75f53c1e2cac', 'request_id': 'req-cea0df24-45bc-4881-898c-d21769b756f7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.434751 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3737c99210b64eac8d7d75f53c1e2cac', 'request_id': 'req-cea0df24-45bc-4881-898c-d21769b756f7'} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.506302 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] The token cache is being invalidate because project 3737c99210b64eac8d7d75f53c1e2cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:48:01.506800 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.508051 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.notifications [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.510932 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1462/2933] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:48:01 2022] DELETE /identity/v3/projects/3737c99210b64eac8d7d75f53c1e2cac => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:48:01.518284 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:01.520248 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:01.536748 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62682a14-2662-40b1-9410-c95d6c5068d0 admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:01.541679 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:48:01.601755 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:01.602814 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:01.603227 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:01.603714 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] PATH_INFO: `/v3/users/f9cd92b0ce17457cae931eca0b6c4c95` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:01.605520 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9cd92b0ce17457cae931eca0b6c4c95)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:01.608948 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:01.825831 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9cd92b0ce17457cae931eca0b6c4c95'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.841540 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9cd92b0ce17457cae931eca0b6c4c95'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.850339 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9cd92b0ce17457cae931eca0b6c4c95'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.921756 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9cd92b0ce17457cae931eca0b6c4c95'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.922387 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9cd92b0ce17457cae931eca0b6c4c95'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:01.929669 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1472/2934] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 00:48:01 2022] DELETE /identity/v3/users/f9cd92b0ce17457cae931eca0b6c4c95 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:48:02.093747 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:02.095913 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:02.181418 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0df24-45bc-4881-898c-d21769b756f7 admin admin] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:02.257512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:02.258512 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:02.259168 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:02.259624 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:02.262318 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:02.282959 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:02.285990 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:02.305699 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1463/2935] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:48:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:48:02.310338 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Authenticating user token {{(pid=88219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:02.312574 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.fernet_utils [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:02.402717 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40af9c-0981-4861-8da9-5c188d145ada admin admin] Validating token access rules against request {{(pid=88219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:02.408938 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b2336264fcc4528b59320d71b7da06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '130b40029b174542ac4b0eacc3caf048', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:02.409573 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:02.409846 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:02.410114 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] PATH_INFO: `/v3/projects/2a64b8377ee74c21b0198dbef64c23c2` {{(pid=88219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:02.411619 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a64b8377ee74c21b0198dbef64c23c2)` {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:02.413568 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] RBAC: Authorization granted {{(pid=88219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:02.528316 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:48:02.542090 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a64b8377ee74c21b0198dbef64c23c2', 'request_id': 'req-f95977ce-3538-4fec-a794-4871092e01c9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:02.621626 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a64b8377ee74c21b0198dbef64c23c2', 'request_id': 'req-f95977ce-3538-4fec-a794-4871092e01c9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:02.630655 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a64b8377ee74c21b0198dbef64c23c2', 'request_id': 'req-f95977ce-3538-4fec-a794-4871092e01c9'} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:02.644209 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] The token cache is being invalidate because project 2a64b8377ee74c21b0198dbef64c23c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:48:02.644530 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:02.645214 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG keystone.notifications [None req-f95977ce-3538-4fec-a794-4871092e01c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:48:02.647005 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: [pid: 88219|app: 0|req: 1473/2936] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 00:48:02 2022] DELETE /identity/v3/projects/2a64b8377ee74c21b0198dbef64c23c2 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:48:12.294146 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=69,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:48:12.673181 dv-cinder-852211-6-iscsi devstack@keystone.service[88219]: DEBUG dbcounter [-] [88219] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:53:06.819410 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] Authenticating user token {{(pid=88218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:53:06.820965 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:53:06.914026 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f30cb-0ff8-4e64-b410-4051a3b96b4a service placement] Validating token access rules against request {{(pid=88218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:53:06.977367 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42f4dfa35f464a3f9cdbbc1e1ee4a980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84d7e2df97aa4867a2b084bd767c48f5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:53:06.977991 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] REQUEST_METHOD: `GET` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:53:06.978269 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] SCRIPT_NAME: `/identity` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:53:06.978538 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:53:06.980445 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.fernet_utils [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:53:06.993685 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:53:06.995307 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b3a0-7fe5-4ce1-a8a3-26a9854008e9 service placement] RBAC: Authorization granted {{(pid=88218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:53:07.007608 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: [pid: 88218|app: 0|req: 1464/2937] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 00:53:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:53:17.002235 dv-cinder-852211-6-iscsi devstack@keystone.service[88218]: DEBUG dbcounter [-] [88218] Writing DB stats keystone:SELECT=63 {{(pid=88218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}